Proposal: Adding consistency to protocol selection

If you run multiple MySQL instances on a Linux machine, chances are good that at one time or another, you’ve ended up connected to an instance other than what you had intended. It’s certainly happened to me, and I submitted Bug#76512 to deal with the cause which affects me most commonly – that the mysql client will silently ignore the –port option and connect using the default Unix socket instead when the host is “localhost” (default).  We’ve recently discussed ways we can make this behavior less surprising to users, and though we’re now past the second RC of MySQL Server 5.7, we’re contemplating making these changes in future 5.7 releases.  Please let us know your thoughts!

Here are the basic principles of what we intend to change:

Explicit –protocol option rules all

If a user provides an explicit –protocol option, the client will only attempt to connect using that protocol.  If it cannot connect using that protocol, it generates an error.  When an explicit –protocol option is supplied, client options related to other protocols will be ignored.  That allows you to have a configuration file for both TCP and Socket connection parameters, while choosing which to use by passing the –protocol command-line option.  For example, this will not happen:

R:\mysql-5.7.8-rc-winx64>bin\mysql -uroot --port=3310 --protocol=TCP --pipe -e"STATUS;"
--------------
bin\mysql  Ver 14.14 Distrib 5.7.8-rc, for Win64 (x86_64)

Connection id:          26
Current database:
Current user:           root@localhost
SSL:                    Not in use
Using delimiter:        ;
Server version:         5.7.8-rc MySQL Community Server (GPL)
Protocol version:       10
Connection:             Named pipe: MySQL
...
--------------

Implicit protocol options

Certain options are associated with a single protocol, and going forward, will implicitly set the protocol.  For example, using the –port option will imply TCP should be used.  This prevents the situation from my bug report where the default socket is used instead of the explicitly requested TCP port, resulting in connection to the wrong instance.  Here are the proposed rules:

  • –host option defined with value other than localhost implies TCP/IP connection
  • –port option defined with any value implies TCP/IP connection
  • –socket option defined with any value implies Unix Socket connection on Linux
  • –socket option defined with any value implies Named Pipe connections on Windows
  • –shared-memory-base-name defined with any value implies Shared Memory connection on Windows

In addition, the current –pipe (or -W) is equivalent to –protocol=TCP, and we propose to deprecate it.

Reject conflicting command-line options

MySQL has historically been very lax in option processing and will generally accept conflicting options instead of producing errors.  For example, you can start the mysql client with all of the above options explicitly defined, without error:

 

R:\mysql-5.7.8-rc-winx64>bin\mysql -uroot --port=3310 --socket=MySQL 
--shared-memory-base-name=MYMEM --pipe
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 27
Server version: 5.7.8-rc MySQL Community Server (GPL)

Which protocol is being used here? It’s using a Named Pipe.

There are some combinations which produce appropriate errors already – most notably when connections are attempted using inter-machine protocols while specifying a remote host:

R:\ade\mysql-5.7.8-rc-winx64>bin\mysql -h192.168.2.2 --protocol=MEMORY
ERROR 2047 (HY000): Wrong or unknown protocol

I think it makes sense to produce similar errors when conflicting implicit command-line options are given. That would prohibit users from starting mysql with both –port and –socket options defined on the command-line.

Retain platform-specific default protocols

When supplying no options that change the behavior, the standard MySQL command-line clients should continue to prefer Unix Socket connections on Linux for performance reasons, while Windows uses TCP/IP.

Conclusion

We think the proposed changes make sense to clarify protocol selection, and that this is a worthwhile addition to 5.7 – even though two Release Candidate builds have already shipped.  With these changes, users (like me) will have better protection against inadvertently connecting to the wrong instance on a multi-instance host, and ambiguity around which protocol is used will be significantly reduced.

Please let us know your thoughts on this proposal!

 

15 thoughts on “Proposal: Adding consistency to protocol selection

  1. It is a splendid idea.
    This change would avoid lots of mysterious errors that leave users baffled. Every time someone complains about a “should-not-happen-here” connection error and I show them the correction by adding or removing one option on the command line, I can see surprise and anger at the current behavior.
    Thanks for introducing this change.

  2. Please don’t make incompatible changes in GA versions. So add this prior to 5.7 GA or don’t add until 5.8. Incompatible changes in minor version updates are a cause of frustration to users as we can’t swap out one GA version and replace with another without checking for such changes. Yes even in client versions. Note your RPM packaging for 5.7 currently forces the client and server versions to be exactly the same which would cause more issues.

    The intent is good, so get this in before going GA. MySQL’s default port is 3306 so it shouldn’t be necessary to provide the port unless it is different, and assume 3306 unless told otherwise.

    I’ve considered having a flag such as allow_incompatible_changes, default 0 to provide such extensions but that’s more on the server side and would allow new features to be added to 5.7 for testing even when their usage would break the compatibility promise. That would help those of us on the edge who want new features while providing compatibility for those who don’t worry about such new technology.

    In the case you are talking about I would just do it, but given you are still pushing in new features into 5.7 the label of RC which implies no more changes ahead of GA was and still seems rather premature.You are adding a lot of good stuff. I was told about feature freezes over a year ago yet changes still come through. So bite the bullet and get 5.7 out so we can all use it and figure out a way to be more agile for 5.8 and later so those who want a stable version get it and those who want these new features can get to use them in production environments.

    1. Hi Simon,

      Thanks for the feedback! I think we’re saying the same thing – once the GA release ships, we won’t want to change behavior here. Just for clarity, though – if the next release turns out to be the first GA release, would you object to having this change in that release (for the first time)?

      The allow_incompatible_changes suggestion is intriguing, and something for us to consider – thanks!

      We’ll keep the existing default values for –port (and –host, etc.). Sorry if that wasn’t clear.

      As always, thanks for the thoughtful feedback!

      1. Oh, so you want to put the new feature in 5.7? I thought we were talking of the next version.

        As a selfish user, I’d love to have that feature ASAP. Really.
        As a QA professional, however, I would be very much worried that such an important change was introduced without adequate time and exposition to the users to test it properly.

        Given the experience with past features, which have never come out clean, I’d say that we need ate least two minor releases before trusting it to a GA.
        Of course, this addition makes the “RC” attribute completely meaningless, but if the feature is added in, and people can try it for a while, I would say that the benefits would be enough to justify it.

        1. Hi Giuseppe,

          That’s completely fair and useful feedback, thanks! We certainly want to avoid destabilizing 5.7 during late RC, and this feature alone surely isn’t worth extending the RC cycle to get a couple of builds out for people to evaluate the change. My thinking is that this change is small enough that we should consider adding it to 5.7 anyways, but that’s exactly why I’m asking for feedback from our community. If such a change undermines confidence in the GA release, we’ll push this change to 5.8 only.

          Thanks again!

  3. I think this is the correct thing to do.

    However I do not like changes in -rc phase. An -rc should be released as GA as-is unless there is some critical bug, then another -rc is needed.

    Maybe make it possible use the 5.8 client with a 5.7 (and 5.6) server (e.g. rpm deps) . Or move the client to it’s own schedule and code base.

    1. Thanks for the feedback Daniël! It’s good to get feedback from the community on the “RC” meaning and what expectations exist.

  4. I think this is a fantastic idea, the sooner it is implemented the better. I have also been burnt a few times when using multiple instances on a single server.

    Will this be backported to older MySQL client versions?

    Thanks,
    Clive

    1. Clive,

      Thanks for the feedback! It seems unlikely at this point to be added to 5.7, given the concerns many in the community have around making such changes this late in the release cycle. We’ll try to get it into 5.8 clients soon, though, and find ways to make these clients more accessible to interested users (Daniël’s point).

  5. Hi Todd,

    I fully support this change.
    Although I like to see this done asap, I am concerned this may be already too late for 5.7…

    Another important point: please make clear error messages, it must be clear why the client can’t connect.

    One last point that you may have considered implicit in your post is the combination of parameters that are passed in different ways to the client, ie. default parameters, client options file and command line options.

    My favourite behaviour would be:
    – reject conflicting command line options
    – reject conflicting options in the options files
    – ignore options that would be conflicting but they are inherited by default or options file.

    For example with the my.cnf file:
    [client]
    port=3316
    protocol=tcp
    socket=myfile.socket

    …“mysql -uroot” should generate an error for conflicting parameters.

    …“mysql -uroot —protocol=tcp” should use the TCP/IP protocol and use port 3316 as specified in the my.cnf file.

    …”mysql -uroot —socket=mysql.sock” should use the Unix socket and ignore the protocol and port options set in my.cnf

    …”mysql -uroot —protocol=socket” should ignore the protocol and port options and use the socket file set in my.cnf (i.e. myfile.socket)

    My 2 cents.
    -ivan

    1. Ivan,

      Thanks for the feedback! I think we’re getting consistent feedback that this is a good proposal, but there’s not a lot of explicit support for introducing it in 5.7, given the current RC state. I thought there might be more interest in getting this into 5.7, but I’m just as happy to target only 5.8 at this time.

      I think we’re largely in agreement on the behavior you describe. My own plans don’t include rejecting conflicting options in the options files – I was only interested in rejecting conflicting command-line options (and ignoring options from files which conflict with explicit command-line options). Practically speaking, this would mean that “mysql -uroot” using the options file you describe above would attempt a TCP/IP connection (because –protocol has priority), rather than generate an error. Do you think that would be acceptable?

      1. I think we should make the behaviour consistent for command line options as well as the options file.

        That said, I appreciate that my point would open a more complex discussion: the command line options are for the mysql client only, the options file can be used by other clients, therefore the check should be in the client lib and not only in the client app. Still my favourite choice though, if you plan it for 5.8, it may be worth to consider to go for the full control of command line _and_ options file.

        1. Thanks Ivan! For my part, the reason I’m interested in allowing conflicting options within an option file is so that a single options file can support multiple connection types, with command-line options used to switch between them. For example:

          [client]
          port=3316
          socket=myfile.socket

          mysql –protocol=TCP
          mysql –protocol=SOCKET

          That said, I appreciate your preference for consistency, and agree it may make sense to move some command-line option handling from the individual clients to libmysql – especially for common options like –protocol and related.

  6. Simon, what’s this about client and server being exactly the same? In a multi-sever setup, it is very important to allow a single client to access any server on the network. In the past, I have encountered virtually no issues — even accessing a 4.1 server with 5.1 client. (Even 4.0 mostly worked.)

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.