ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Port from SW - Salt master rsa key issue

    IT Discussion
    salt salt master salt minion rsa
    6
    60
    9.5k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • scottalanmillerS
      scottalanmiller
      last edited by

      @dgingerich said in Port from SW - Salt master rsa key issue:

      [WARNING ] Key 'file_ignore_glob' with value None has an invalid type of NoneType, a list is required for this value

      maybe you have duplicate DNS entries and round robin is getting you?

      1 Reply Last reply Reply Quote 0
      • D
        dgingerich
        last edited by

        @scottalanmiller said in Port from SW - Salt master rsa key issue:

        @dgingerich said in Port from SW - Salt master rsa key issue:

        [WARNING ] Key 'file_ignore_glob' with value None has an invalid type of NoneType, a list is required for this value

        maybe you have duplicate DNS entries and round robin is getting you?

        The DNS is just one address for each. I changed the to the new IPs while the systems were building at the authoritative servers. (Systems are housed at Packet.net, DNS handled through AWS) So, there should not be any DNS caching issues.

        scottalanmillerS 1 Reply Last reply Reply Quote 0
        • scottalanmillerS
          scottalanmiller @dgingerich
          last edited by

          @dgingerich said in Port from SW - Salt master rsa key issue:

          @scottalanmiller said in Port from SW - Salt master rsa key issue:

          @dgingerich said in Port from SW - Salt master rsa key issue:

          [WARNING ] Key 'file_ignore_glob' with value None has an invalid type of NoneType, a list is required for this value

          maybe you have duplicate DNS entries and round robin is getting you?

          The DNS is just one address for each. I changed the to the new IPs while the systems were building at the authoritative servers. (Systems are housed at Packet.net, DNS handled through AWS) So, there should not be any DNS caching issues.

          Oh okay, this is all hosted. Still, best to be sure and rule out possibilities while testing. This is weird, we use Vultr for Salt Masters and have never seen anything like this. But we avoid Ubuntu, so if there is any bug there, we'd not have seen it.

          D 1 Reply Last reply Reply Quote 0
          • D
            dgingerich @scottalanmiller
            last edited by

            @scottalanmiller said in Port from SW - Salt master rsa key issue:

            @dgingerich said in Port from SW - Salt master rsa key issue:

            @scottalanmiller said in Port from SW - Salt master rsa key issue:

            @dgingerich said in Port from SW - Salt master rsa key issue:

            [WARNING ] Key 'file_ignore_glob' with value None has an invalid type of NoneType, a list is required for this value

            maybe you have duplicate DNS entries and round robin is getting you?

            The DNS is just one address for each. I changed the to the new IPs while the systems were building at the authoritative servers. (Systems are housed at Packet.net, DNS handled through AWS) So, there should not be any DNS caching issues.

            Oh okay, this is all hosted. Still, best to be sure and rule out possibilities while testing. This is weird, we use Vultr for Salt Masters and have never seen anything like this. But we avoid Ubuntu, so if there is any bug there, we'd not have seen it.

            I was able to build another system, named QAICS-mastertest, that worked perfectly using the exact same methods. It's really weird.

            scottalanmillerS 1 Reply Last reply Reply Quote 1
            • scottalanmillerS
              scottalanmiller @dgingerich
              last edited by

              @dgingerich said in Port from SW - Salt master rsa key issue:

              @scottalanmiller said in Port from SW - Salt master rsa key issue:

              @dgingerich said in Port from SW - Salt master rsa key issue:

              @scottalanmiller said in Port from SW - Salt master rsa key issue:

              @dgingerich said in Port from SW - Salt master rsa key issue:

              [WARNING ] Key 'file_ignore_glob' with value None has an invalid type of NoneType, a list is required for this value

              maybe you have duplicate DNS entries and round robin is getting you?

              The DNS is just one address for each. I changed the to the new IPs while the systems were building at the authoritative servers. (Systems are housed at Packet.net, DNS handled through AWS) So, there should not be any DNS caching issues.

              Oh okay, this is all hosted. Still, best to be sure and rule out possibilities while testing. This is weird, we use Vultr for Salt Masters and have never seen anything like this. But we avoid Ubuntu, so if there is any bug there, we'd not have seen it.

              I was able to build another system, named QAICS-mastertest, that worked perfectly using the exact same methods. It's really weird.

              Very weird. So the changing of the name played a role.

              1 Reply Last reply Reply Quote 0
              • D
                dgingerich
                last edited by

                OK. I've deleted the DNS entries and the systems from Packet.net. I'm going to try again tomorrow morning after all the DNS caching should have expired.

                scottalanmillerS 1 Reply Last reply Reply Quote 1
                • scottalanmillerS
                  scottalanmiller @dgingerich
                  last edited by

                  @dgingerich said in Port from SW - Salt master rsa key issue:

                  OK. I've deleted the DNS entries and the systems from Packet.net. I'm going to try again tomorrow morning after all the DNS caching should have expired.

                  Okay. I'll be around.

                  1 Reply Last reply Reply Quote 0
                  • D
                    dgingerich
                    last edited by

                    Thank you very much for your time on this.

                    scottalanmillerS 1 Reply Last reply Reply Quote 0
                    • scottalanmillerS
                      scottalanmiller @dgingerich
                      last edited by

                      @dgingerich said in Port from SW - Salt master rsa key issue:

                      Thank you very much for your time on this.

                      No problem. Sorry that we've not gotten it all figured out yet.

                      1 Reply Last reply Reply Quote 0
                      • Emad RE
                        Emad R
                        last edited by Emad R

                        I too, not an expert, more like playing with salt and you seem like you know more about it than me, but this one liner helps me when I feel something is cached in the setting, or command fails cause its already running:

                        salt '*' saltutil.kill_all_jobs && salt-run cache.clear_all && salt '*' saltutil.clear_cache && salt '*' saltutil.sync_all
                        
                        1 Reply Last reply Reply Quote 1
                        • D
                          dgingerich
                          last edited by

                          @msff-amman-Itofficer said in Port from SW - Salt master rsa key issue:

                          I too, not an expert, more like playing with salt and you seem like you know more about it than me, but this one liner helps me when I feel something is cached in the setting, or command fails cause its already running:

                          salt '*' saltutil.kill_all_jobs && salt-run cache.clear_all && salt '*' saltutil.clear_cache && salt '*' saltutil.sync_all
                          

                          Yeah, that wouldn't work because the masters simply aren't talking to the minions.

                          Over the weekend, I tried to delete the DNS and recreate the masters after 12 hours, and that did sort of work, for a bit. The masters would talk to themselves, but as soon as two minions were connected, the communication just stopped again. I don't have any idea what is causing this.

                          1 Reply Last reply Reply Quote 0
                          • scottalanmillerS
                            scottalanmiller
                            last edited by

                            What if you create new masters with totally new names. Start fresh, including the names. A huge pain, I know, but I think that that key regeneration something horrible. Just avoid that next time, use the keys that are generated automatically only. This is a bad break that I'd love to get figured out, but I know that you have a time crunch and we want this working, too.

                            D 1 Reply Last reply Reply Quote 0
                            • D
                              dgingerich @scottalanmiller
                              last edited by

                              This post is deleted!
                              D 1 Reply Last reply Reply Quote 0
                              • D
                                dgingerich @dgingerich
                                last edited by

                                @dgingerich

                                @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                What if you create new masters with totally new names. Start fresh, including the names. A huge pain, I know, but I think that that key regeneration something horrible. Just avoid that next time, use the keys that are generated automatically only. This is a bad break that I'd love to get figured out, but I know that you have a time crunch and we want this working, too.

                                Tried this. It works, but the salt scripts have to be adjusted for the new names in order to work, and I have not had success with that. The guy who made the scripts is in Italy this whole week, and then UK for the week after that.

                                scottalanmillerS 1 Reply Last reply Reply Quote 0
                                • scottalanmillerS
                                  scottalanmiller @dgingerich
                                  last edited by

                                  @dgingerich said in Port from SW - Salt master rsa key issue:

                                  @dgingerich

                                  @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                  What if you create new masters with totally new names. Start fresh, including the names. A huge pain, I know, but I think that that key regeneration something horrible. Just avoid that next time, use the keys that are generated automatically only. This is a bad break that I'd love to get figured out, but I know that you have a time crunch and we want this working, too.

                                  Tried this. It works, but the salt scripts have to be adjusted for the new names in order to work, and I have not had success with that. The guy who made the scripts is in Italy this whole week, and then UK for the week after that.

                                  That should just be a matter of putting them into the top.sls file.

                                  D 1 Reply Last reply Reply Quote 0
                                  • D
                                    dgingerich @scottalanmiller
                                    last edited by

                                    @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                    @dgingerich said in Port from SW - Salt master rsa key issue:

                                    @dgingerich

                                    @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                    What if you create new masters with totally new names. Start fresh, including the names. A huge pain, I know, but I think that that key regeneration something horrible. Just avoid that next time, use the keys that are generated automatically only. This is a bad break that I'd love to get figured out, but I know that you have a time crunch and we want this working, too.

                                    Tried this. It works, but the salt scripts have to be adjusted for the new names in order to work, and I have not had success with that. The guy who made the scripts is in Italy this whole week, and then UK for the week after that.

                                    That should just be a matter of putting them into the top.sls file.

                                    There's a lot of customization in the iptables config and other areas that are scripted so that future systems could be deployed quickly.

                                    We have been ordered by management to go ahead with a manual build on the QA stack. So, the idea of using salt in this case is out.

                                    scottalanmillerS 1 Reply Last reply Reply Quote 0
                                    • scottalanmillerS
                                      scottalanmiller @dgingerich
                                      last edited by

                                      @dgingerich said in Port from SW - Salt master rsa key issue:

                                      There's a lot of customization in the iptables config and other areas that are scripted so that future systems could be deployed quickly.

                                      If that was handled in Salt, it would be all automated so that it would solve this problem, rather than create it. That's actually a reason to use Salt in that case.

                                      D 1 Reply Last reply Reply Quote 1
                                      • D
                                        dgingerich @scottalanmiller
                                        last edited by

                                        @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                        @dgingerich said in Port from SW - Salt master rsa key issue:

                                        There's a lot of customization in the iptables config and other areas that are scripted so that future systems could be deployed quickly.

                                        If that was handled in Salt, it would be all automated so that it would solve this problem, rather than create it. That's actually a reason to use Salt in that case.

                                        The production stack has Salt working. It's the QA stack that doesn't. The production stack is 62 servers, and needs Salt. The QA stack is just 8 servers plus 2 management servers, and the salt config is supposed to be the same, but adjusted for other system names. I don't have the Salt skills to adjust the Salt scripts to work for the QA stack.

                                        scottalanmillerS 1 Reply Last reply Reply Quote 0
                                        • scottalanmillerS
                                          scottalanmiller @dgingerich
                                          last edited by

                                          @dgingerich said in Port from SW - Salt master rsa key issue:

                                          @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                          @dgingerich said in Port from SW - Salt master rsa key issue:

                                          There's a lot of customization in the iptables config and other areas that are scripted so that future systems could be deployed quickly.

                                          If that was handled in Salt, it would be all automated so that it would solve this problem, rather than create it. That's actually a reason to use Salt in that case.

                                          The production stack has Salt working. It's the QA stack that doesn't. The production stack is 62 servers, and needs Salt. The QA stack is just 8 servers plus 2 management servers, and the salt config is supposed to be the same, but adjusted for other system names. I don't have the Salt skills to adjust the Salt scripts to work for the QA stack.

                                          Hmmm... might want to pressure your Salt guy to automate that. In theory, you could have something like QA in the name that designates that for the future.

                                          D 1 Reply Last reply Reply Quote 1
                                          • D
                                            dgingerich @scottalanmiller
                                            last edited by

                                            @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                            @dgingerich said in Port from SW - Salt master rsa key issue:

                                            @scottalanmiller said in Port from SW - Salt master rsa key issue:

                                            @dgingerich said in Port from SW - Salt master rsa key issue:

                                            There's a lot of customization in the iptables config and other areas that are scripted so that future systems could be deployed quickly.

                                            If that was handled in Salt, it would be all automated so that it would solve this problem, rather than create it. That's actually a reason to use Salt in that case.

                                            The production stack has Salt working. It's the QA stack that doesn't. The production stack is 62 servers, and needs Salt. The QA stack is just 8 servers plus 2 management servers, and the salt config is supposed to be the same, but adjusted for other system names. I don't have the Salt skills to adjust the Salt scripts to work for the QA stack.

                                            Hmmm... might want to pressure your Salt guy to automate that. In theory, you could have something like QA in the name that designates that for the future.

                                            Yeah, the reason I'm in the middle of all this is because our Salt guy is on his honeymoon in Italy this week, and the UK next week. So, he's not going to be of much help for a while.

                                            scottalanmillerS 1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 3 / 3
                                            • First post
                                              Last post