ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Building ELK on CentOS 7

    IT Discussion
    scale ntg lab scale hc3 centos centos 7 elk logging log management how to linux elasticsearch kibana logstash kibana 4
    8
    43
    16.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JaredBuschJ
      JaredBusch
      last edited by

      like this

      [root@elk ~]# df -h
      Filesystem                   Size  Used Avail Use% Mounted on
      /dev/mapper/centos_elk-root   50G  855M   50G   2% /
      devtmpfs                     906M     0  906M   0% /dev
      tmpfs                        916M     0  916M   0% /dev/shm
      tmpfs                        916M  8.3M  907M   1% /run
      tmpfs                        916M     0  916M   0% /sys/fs/cgroup
      /dev/sda2                    494M   98M  396M  20% /boot
      /dev/sda1                    200M  9.5M  191M   5% /boot/efi
      /dev/mapper/centos_elk-home   75G   33M   75G   1% /home
      tmpfs                        184M     0  184M   0% /run/user/0
      [root@elk ~]#
      
      1 Reply Last reply Reply Quote 0
      • JaredBuschJ
        JaredBusch
        last edited by

        You should at least tell the user that you are asking for the kibana password.

        htpasswd -c /etc/nginx/htpasswd.users kibanauser

        0_1456293132171_upload-2a181928-c672-4286-86d6-43b69bd92fc3

        1 Reply Last reply Reply Quote 0
        • JaredBuschJ
          JaredBusch
          last edited by

          I had this error.

          0_1456293268870_upload-fc8bbfa4-c8e3-4c86-8e10-5cd5cf3195c8

          1 Reply Last reply Reply Quote 0
          • JaredBuschJ
            JaredBusch
            last edited by

            Looks like maybe you forgot to start firewalld?

              % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                             Dload  Upload   Total   Spent    Left  Speed
            100   814  100   814    0     0   1370      0 --:--:-- --:--:-- --:--:--  1372
            {
              "acknowledged" : true
            }
            FirewallD is not running
            FirewallD is not running
            [root@elk ~]# yum install firewalld
            Loaded plugins: fastestmirror
            Loading mirror speeds from cached hostfile
             * base: mirror.oss.ou.edu
             * epel: fedora-epel.mirror.lstn.net
             * extras: centos.mirrors.wvstateu.edu
             * updates: centos.mirrors.wvstateu.edu
            Package firewalld-0.3.9-14.el7.noarch already installed and latest version
            Nothing to do
            [root@elk ~]# systemctl start firewalld
            [root@elk ~]# systemctl status firewalld
            ● firewalld.service - firewalld - dynamic firewall daemon
               Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
               Active: active (running) since Tue 2016-02-23 23:55:11 CST; 14s ago
             Main PID: 11482 (firewalld)
               CGroup: /system.slice/firewalld.service
                       └─11482 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
            
            Feb 23 23:55:09 elk systemd[1]: Starting firewalld - dynamic firewall daemon...
            Feb 23 23:55:11 elk systemd[1]: Started firewalld - dynamic firewall daemon.
            [root@elk ~]#
            
            scottalanmillerS 1 Reply Last reply Reply Quote 0
            • JaredBuschJ
              JaredBusch
              last edited by

              Yeah, you set it to install, but you never start or enable it.

              0_1456293489801_upload-2c71d7be-5435-4f22-ab21-c2912aa344c8

              1 Reply Last reply Reply Quote 0
              • JaredBuschJ
                JaredBusch
                last edited by JaredBusch

                Line 109 needs commented out.

                0_1456293589646_upload-722d8a55-ede0-467f-815e-97aca00bde17

                add this right after the yum install to fix the firewall.

                yum -y install wget firewalld epel-release
                systemctl enable firewalld
                systemctl start firewalld
                yum -y install nginx httpd-tools unzip
                

                I would just remove line 109 it serves no purpose.

                Edit: Some dumbass forgot to snapshot the image so he could repeat the install...

                scottalanmillerS 1 Reply Last reply Reply Quote 0
                • JaredBuschJ
                  JaredBusch
                  last edited by JaredBusch

                  Why lock out with .htaccess? There is no hint what is needed to log in here.

                  0_1456293980571_upload-16198c5d-89fa-48ac-a702-3d6b2cc05644

                  I hate this level of authentication.

                  Using kibanauser and the password I chose, results in Kibana setup.
                  0_1456294107205_upload-68bec54b-23aa-4026-95d9-8080cfed408d

                  scottalanmillerS 1 Reply Last reply Reply Quote 0
                  • scottalanmillerS
                    scottalanmiller @JaredBusch
                    last edited by

                    @JaredBusch said:

                    @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                    If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                    JaredBuschJ 1 Reply Last reply Reply Quote 0
                    • scottalanmillerS
                      scottalanmiller @JaredBusch
                      last edited by

                      @JaredBusch said:

                      Why lock out with .htaccess? There is no hint what is needed to log in here.

                      It's how Digital Ocean does it as well. Kibana doesn't have a built in authentication scheme that I know of. HTAccess is very simple for someone to just get started.

                      1 Reply Last reply Reply Quote 0
                      • scottalanmillerS
                        scottalanmiller
                        last edited by

                        And simple to remove when you want to move to something else.

                        1 Reply Last reply Reply Quote 0
                        • scottalanmillerS
                          scottalanmiller @JaredBusch
                          last edited by

                          @JaredBusch said:

                          Line 109 needs commented out.

                          0_1456293589646_upload-722d8a55-ede0-467f-815e-97aca00bde17

                          add this right after the yum install to fix the firewall.

                          yum -y install wget firewalld epel-release
                          systemctl enable firewalld
                          systemctl start firewalld
                          yum -y install nginx httpd-tools unzip
                          

                          I would just remove line 109 it serves no purpose.

                          Edit: Some dumbass forgot to snapshot the image so he could repeat the install...

                          Thanks. That was formatting I had originally put in before scripting it.

                          1 Reply Last reply Reply Quote 0
                          • scottalanmillerS
                            scottalanmiller @JaredBusch
                            last edited by

                            @JaredBusch said:

                            Looks like maybe you forgot to start firewalld?

                            Fixed

                            1 Reply Last reply Reply Quote 0
                            • JaredBuschJ
                              JaredBusch @scottalanmiller
                              last edited by

                              @scottalanmiller said:

                              @JaredBusch said:

                              @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                              If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                              SO this mean you need to make one of your linux admin setrups on drive settings because that is not what CentOS does by dfault.

                              DashrenderD 1 Reply Last reply Reply Quote 0
                              • DashrenderD
                                Dashrender @JaredBusch
                                last edited by

                                @JaredBusch said:

                                @scottalanmiller said:

                                @JaredBusch said:

                                @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                                If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                                SO this mean you need to make one of your linux admin setrups on drive settings because that is not what CentOS does by dfault.

                                Would CentOS do what Scott does if you had two drives you provide CentOS to use? i.e. a 20 GB and a 200+ GB one? Would CentOS install the OS and everything fully on the 20, and then just mount the 200 on some point?

                                JaredBuschJ 1 Reply Last reply Reply Quote 0
                                • JaredBuschJ
                                  JaredBusch @Dashrender
                                  last edited by

                                  @Dashrender said:

                                  @JaredBusch said:

                                  @scottalanmiller said:

                                  @JaredBusch said:

                                  @scottalanmiller so what do you setup your disk partitioning like in CentOS 7?

                                  If I'm doing this for product, I do 20GB for the OS and 200GB+ on a second VHD for the data. I put it all under LVM and make a XFS filesystem on the secondary mount and mount it to data and make a symlink for the Elasticsearch database directory into there.

                                  SO this mean you need to make one of your linux admin setrups on drive settings because that is not what CentOS does by dfault.

                                  Would CentOS do what Scott does if you had two drives you provide CentOS to use? i.e. a 20 GB and a 200+ GB one? Would CentOS install the OS and everything fully on the 20, and then just mount the 200 on some point?

                                  Than answer is not by default. It tries to make it's own magic.

                                  You can see here I created a 20gb and a 200GB vhdx and told the install to handle it all for me.

                                  0_1456324763628_upload-a8516602-b5e4-4c0e-9112-caabbb970b80

                                  Guess what, you still end up with a 50GB and a 170GB partitions scheme

                                  [root@elk ~]# df -h
                                  Filesystem                   Size  Used Avail Use% Mounted on
                                  /dev/mapper/centos_elk-root   50G  882M   50G   2% /
                                  devtmpfs                     906M     0  906M   0% /dev
                                  tmpfs                        916M     0  916M   0% /dev/shm
                                  tmpfs                        916M  8.3M  907M   1% /run
                                  tmpfs                        916M     0  916M   0% /sys/fs/cgroup
                                  /dev/sda2                    494M   99M  395M  21% /boot
                                  /dev/sda1                    200M  9.5M  191M   5% /boot/efi
                                  /dev/mapper/centos_elk-home  168G   33M  168G   1% /home
                                  tmpfs                        184M     0  184M   0% /run/user/0
                                  [root@elk ~]#
                                  
                                  1 Reply Last reply Reply Quote 0
                                  • JaredBuschJ
                                    JaredBusch
                                    last edited by

                                    CentOS 7 has a thing for 50GB root mounts.

                                    1 Reply Last reply Reply Quote 0
                                    • scottalanmillerS
                                      scottalanmiller
                                      last edited by

                                      Yeah, the defaults suck a bit.

                                      1 Reply Last reply Reply Quote 0
                                      • JaredBuschJ
                                        JaredBusch
                                        last edited by JaredBusch

                                        @scottalanmiller Why are you using Oracle's Java SDK and not just java from the repo?

                                        I had read in another write up on the install that it works fine even if it is not the "official" method.

                                        scottalanmillerS 1 Reply Last reply Reply Quote 0
                                        • scottalanmillerS
                                          scottalanmiller @JaredBusch
                                          last edited by

                                          @JaredBusch said:

                                          @scottalanmiller Why are you using Oracle's Java SDK and not just java from the repo?

                                          I had read in another write up on the install that it works fine even if it is not the "official" method.

                                          Even if it "works", Elasticsearch tests against and only officially supports the Oracle one. Just because I can get the OpenJDK to work, I'd hate to have it be buggy or problematic for someone down the line because I used one that wasn't tested against.

                                          JaredBuschJ 1 Reply Last reply Reply Quote 1
                                          • JaredBuschJ
                                            JaredBusch @scottalanmiller
                                            last edited by

                                            @scottalanmiller said:

                                            @JaredBusch said:

                                            @scottalanmiller Why are you using Oracle's Java SDK and not just java from the repo?

                                            I had read in another write up on the install that it works fine even if it is not the "official" method.

                                            Even if it "works", Elasticsearch tests against and only officially supports the Oracle one. Just because I can get the OpenJDK to work, I'd hate to have it be buggy or problematic for someone down the line because I used one that wasn't tested against.

                                            I am not a fan of Oracle when it comes to Java

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 1 / 3
                                            • First post
                                              Last post