ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Solved Need to parse large conf files

    IT Discussion
    scripting asterisk config
    6
    53
    3.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 1
      1337 @JaredBusch
      last edited by

      @JaredBusch said in Need to parse large conf files:

      @matteo-nunziati said in Need to parse large conf files:

      @JaredBusch said in Need to parse large conf files:

      The catching the exten => STUFF,1 is the harder part. it HAS to be that full syntaxt.

      no it is simple with regex IF the line is:

      exten => bla,1,bla

      100% those are the only exten lines I want.

      Is it possible to have exten => bla,2,bla ??

      J 1 Reply Last reply Reply Quote 0
      • J
        JaredBusch @1337
        last edited by

        @Pete-S said in Need to parse large conf files:

        @JaredBusch said in Need to parse large conf files:

        @matteo-nunziati said in Need to parse large conf files:

        @JaredBusch said in Need to parse large conf files:

        The catching the exten => STUFF,1 is the harder part. it HAS to be that full syntaxt.

        no it is simple with regex IF the line is:

        exten => bla,1,bla

        100% those are the only exten lines I want.

        Is it possible to have exten => bla,2,bla ??

        Yes, but I do not want them.

        1 2 Replies Last reply Reply Quote 0
        • M
          matteo nunziati @JaredBusch
          last edited by

          @JaredBusch said in Need to parse large conf files:

          @matteo-nunziati said in Need to parse large conf files:

          @JaredBusch said in Need to parse large conf files:

          The catching the exten => STUFF,1 is the harder part. it HAS to be that full syntaxt.

          no it is simple with regex IF the line is:

          exten => bla,1,bla

          100% those are the only exten lines I want.

          @Pete-S if the php code works for @JaredBusch , just substitute the search part with the provided regex.

          1 Reply Last reply Reply Quote 1
          • 1
            1337 @JaredBusch
            last edited by

            @JaredBusch Could you check that you can run command line php on the server?
            Just execute php -v and you'll get the version.

            At least on debian you need to have php-cli package and I don't know if you do.

            J 1 Reply Last reply Reply Quote 0
            • 1
              1337 @JaredBusch
              last edited by 1337

              @JaredBusch
              Also is it just one conf file you need to search through or many in the same directory ?

              J 1 Reply Last reply Reply Quote 0
              • J
                JaredBusch @1337
                last edited by

                @Pete-S said in Need to parse large conf files:

                @JaredBusch Could you check that you can run command line php on the server?
                Just execute php -v and you'll get the version.

                At least on debian you need to have php-cli package and I don't know if you do.

                I can.

                1 Reply Last reply Reply Quote 0
                • J
                  JaredBusch @1337
                  last edited by

                  @Pete-S said in Need to parse large conf files:

                  @JaredBusch
                  Also is it just one conf file you need to search through or many in the same directory ?

                  many. but feeding that into a loop is simple once I work this out.

                  1 1 Reply Last reply Reply Quote 0
                  • 1
                    1337 @JaredBusch
                    last edited by 1337

                    @JaredBusch said in Need to parse large conf files:

                    @Pete-S said in Need to parse large conf files:

                    @JaredBusch
                    Also is it just one conf file you need to search through or many in the same directory ?

                    many. but feeding that into a loop is simple once I work this out.

                    I had file scanning already in the example file but can take it out. The reason I'm asking is if you need to have the filename too in the "report" for each extension?

                    J 1 Reply Last reply Reply Quote 0
                    • J
                      JaredBusch @1337
                      last edited by

                      @Pete-S said in Need to parse large conf files:

                      @JaredBusch said in Need to parse large conf files:

                      @Pete-S said in Need to parse large conf files:

                      @JaredBusch
                      Also is it just one conf file you need to search through or many in the same directory ?

                      many. but feeding that into a loop is simple once I work this out.

                      I had file scanning already in the example file but can take it out. The reason I'm asking is if you need to have the filename too in the "report" for each extension?

                      Yes. but why i dunno. this is all a hand coded mess when compared to standards.

                      It is all perfectly valid syntax. and there are comments all over. but jsut wow.

                      1 Reply Last reply Reply Quote 0
                      • M
                        matteo nunziati
                        last edited by matteo nunziati

                        ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S

                        grep -B3 -n -e "^exten\|^[" $YOURFILENAME | sort -n | uniq | cut -f2 -d: | cut -f2 -d- | grep -e "^[e[\;]"
                        

                        still not granted to get only valid exten

                        J M 2 Replies Last reply Reply Quote 0
                        • J
                          JaredBusch @matteo nunziati
                          last edited by

                          @matteo-nunziati said in Need to parse large conf files:

                          ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S

                          I'm not waiting. I'm doing my thing too.

                          1 Reply Last reply Reply Quote 1
                          • M
                            matteo nunziati @matteo nunziati
                            last edited by matteo nunziati

                            @matteo-nunziati said in Need to parse large conf files:

                            ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S

                            grep -B3 -n -e "^exten\|^[" $YOURFILENAME | sort -n | uniq | cut -f2 -d: | cut -f2 -d- | grep -e "^[e[\;]"
                            

                            still not granted to get only valid exten

                            this is the outcome for @Pete-S test file:

                            ;; This is a test file
                            [context_1]
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            ;;   Ben's inbound 2344242342                      ;
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            exten => 2344242342,1,NoOp()
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            ;;   Bob's inbound 3145551212                      ;
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            exten => 3145551212,1,NoOp()
                            ;;  Mary's inbound 4534535345                      ;
                            ;   With some added comments                       ;
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            exten => 4534535345,1,NoOp()
                            [context_2]
                            exten => 33333333,1,NoOp()
                            ;    Line 3,,,                                     ;
                            ;    Line 4;Let's see if that comment remains      ;
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            exten => 3145454,1,NoOp()
                            ;   And lets add this too                         ;;
                            ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                            exten => 232342,1,NoOp()
                            [context_3]
                            exten => 7777777,1
                            exten => 8888888,1,NoOp()
                            
                            J 1 Reply Last reply Reply Quote 0
                            • J
                              JaredBusch @matteo nunziati
                              last edited by

                              @matteo-nunziati said in Need to parse large conf files:

                              ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S
                              still not granted to get only valid exten

                              doens't work right on a live file. close. but not correct.

                              1 Reply Last reply Reply Quote 0
                              • J
                                JaredBusch @JaredBusch
                                last edited by JaredBusch

                                @JaredBusch said in Need to parse large conf files:

                                // find matching files, put in array called filename
                                $filename:=glob("*.ini");

                                it doesn't like the :

                                I moved a copy of the files to the FreePBX 14 box since PHP 5.6 is "current" for CentOS.

                                [jbusch@cccfreepbx newton_conf]$ sudo -u asterisk php parseit.php 
                                PHP Parse error:  syntax error, unexpected ':' in /var/www/html/newton_conf/parseit.php on line 3
                                

                                2821a8fe-1c2b-4c43-9f87-68a5baecf3aa-image.png

                                3817d8e0-37a5-4835-870f-0158ff0cd659-image.png

                                1 2 Replies Last reply Reply Quote 0
                                • 1
                                  1337 @JaredBusch
                                  last edited by

                                  @JaredBusch

                                  Hang on a sec. I'm just about finished.

                                  1 2 Replies Last reply Reply Quote 0
                                  • 1
                                    1337 @1337
                                    last edited by 1337

                                    This is the output:

                                    File: sample.conf
                                    [context_1]
                                    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                                    ;;   Bob's inbound 3145551212                      ;
                                    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                                    exten => 3145551212,1,NoOp()
                                    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                                    ;;  Mary's inbound 4534535345                      ;
                                    ;   With some added comments                       ;
                                    exten => 4534535345,1,NoOp()
                                    [context_2]
                                    exten => 33333333,1,NoOp()
                                    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                                    ;;   What is this????????????                      ;
                                    ;    Lets add a long comment section               ;
                                    exten => 3145454,1,NoOp()
                                    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                                    ;;  Let's make a wonky comment line here
                                    ;   And lets add this too                         ;;
                                    exten => 232342,1,NoOp()
                                    [context_3]
                                    exten => 7777777,1
                                    exten => 8888888,1,NoOp()
                                    

                                    Personally I'd format it a little for instance by remove all comment lines that are just ;;;;;; and all trailing and preceding ;

                                    Or if it's just for viewing I'd html it.

                                    1 Reply Last reply Reply Quote 0
                                    • 1
                                      1337 @1337
                                      last edited by

                                      This is the complete php file with comments.

                                      It takes all *.conf files and goes through them.

                                      <?php
                                      
                                         // find matching files, put in array called filename
                                         $filename=glob("*.conf");
                                         
                                      
                                         // scan through all the files
                                         foreach ($filename as $f) {
                                            // print filename
                                            print "File: $f\n";
                                      
                                            // read all lines into lines array
                                            $lines=file($f);
                                      
                                            $no=0; // linenumber
                                            $comment=[]; // comments array
                                      
                                            // go through the file and pick out what we need
                                            foreach ($lines as $line) {
                                               $no++;
                                      
                                               // look for [context]
                                               if ($line[0]=='[') {
                                                  $context=$line;
                                                  print "$context";
                                               }
                                               
                                               // look for extensions
                                               $search="exten =>";
                                               if (substr($line,0,strlen($search))==$search) {
                                                  //print $line;
                                      
                                                  // we found the extension row
                                                  // lets take everything after "exten =>"
                                                  $extline=trim(substr($line,strlen($search)));
                                      
                                                  // lets divide it up
                                                  $parts=explode(',', $extline);
                                                  $ext=$parts[0]; // extension number
                                      
                                                  // check that it's ?????,1,?????
                                                  if ($parts[1]=='1') {
                                                     // valid extension, actual extension is $parts[0];
                                                     $ext=$line;
                                                     // print comments
                                                     foreach($comment as $c) print "$c";
                                                     // print the extesion and empty line
                                                     print "$ext";
                                                  }
                                                  $comments=[]; // clear comments
                                               }
                                               
                                               // look for comments
                                               if ($line[0]==';') {
                                                  // add to comments but only first 3 lines
                                                  if (count($comment)<3) $comment[]=$line;
                                                  //print_r($comment);
                                               } else {
                                      
                                                  // non empty line? => clear comments
                                                  if (trim($line)>'') $comment=[];
                                      
                                               }
                                            }
                                         }
                                      
                                      ?>
                                      
                                      1 Reply Last reply Reply Quote 0
                                      • 1
                                        1337 @JaredBusch
                                        last edited by 1337

                                        @JaredBusch said in Need to parse large conf files:

                                        @JaredBusch said in Need to parse large conf files:

                                        // find matching files, put in array called filename
                                        $filename:=glob("*.ini");

                                        it doesn't like the :

                                        I moved a copy of the files to the FreePBX 14 box since PHP 5.6 is "current" for CentOS.

                                        It's because I've been working with another language all day that uses := to assign a variable and not just =

                                        Takes me a few minutes to reboot my brain into php mode. :smiling_face_with_smiling_eyes:

                                        1 1 Reply Last reply Reply Quote 0
                                        • 1
                                          1337 @1337
                                          last edited by 1337

                                          Just for fun I found some extensions.conf file on the net to try. It led me to add some better formatting so the output is more readable.

                                          PS. Just a small change like this that would have caused hours of search on regex, options for grep, awk, sed etc to try and make something work. That's why I think it's easier to just go full on programming language as soon as it is more complex than just finding a few lines that matches.

                                          I know as this is all for naught but whatever.

                                          Output from the php script:

                                          File: sample.conf
                                          
                                          [context_1]
                                          ; Bob's inbound 3145551212
                                          exten => 3145551212,1,NoOp()
                                          
                                          ; Mary's inbound 4534535345
                                          ; With some added comments
                                          exten => 4534535345,1,NoOp()
                                          
                                          [context_2]
                                          exten => 33333333,1,NoOp()
                                          
                                          ; What is this????????????
                                          ; Lets add a long comment section
                                          ; Line 3,,,
                                          exten => 3145454,1,NoOp()
                                          
                                          ; Let's make a wonky comment line here
                                          ; And lets add this too
                                          exten => 232342,1,NoOp()
                                          
                                          [context_3]
                                          exten => 7777777,1
                                          
                                          exten => 8888888,1,NoOp()
                                          

                                          PHP script v2

                                          <?php
                                             // Ver 2 - improved formatting, only showing headings that have extensions
                                             
                                             // find matching files, put in array called filename
                                             $filename=glob("*.conf");
                                             
                                          
                                             // scan through all the files
                                             foreach ($filename as $f) {
                                                // print filename
                                                print "File: $f\n\n";
                                          
                                                // read all lines into lines array
                                                $lines=file($f);
                                          
                                                $no=0; // linenumber
                                                $comment=[]; // comments array
                                                $context=''; // context text
                                          
                                                // go through the file and pick out what we need
                                                foreach ($lines as $line) {
                                                   $no++;
                                          
                                                   // look for [context]
                                                   if ($line[0]=='[') {
                                                      $context=trim($line);
                                                   }
                                                   
                                                   // look for extensions
                                                   $search="exten =>";
                                                   if (substr($line,0,strlen($search))==$search) {
                                                      //print $line;
                                          
                                                      // we found the extension row
                                                      // lets take everything after "exten =>"
                                                      $extline=trim(substr($line,strlen($search)));
                                          
                                                      // lets divide it up
                                                      $parts=explode(',', $extline);
                                          
                                                      // check that it's ?????,1,?????
                                                      if ($parts[1]=='1') {
                                                         // valid extension, actual extension is $parts[0];
                                                         $ext=trim($line);
                                                         
                                                         // print context if it has not been printed
                                                         if ($context>'') { print "$context\n"; $context=''; }
                                                         // print comments
                                                         foreach($comment as $c) print "; $c\n";
                                                         // print the extension and empty line
                                                         print "$ext\n\n";
                                                      }
                                                      $comments=[]; // clear comments
                                                   }
                                                   
                                                   // look for comments
                                                   if ($line[0]==';') {
                                                      // remove whitespace and ;
                                                      $s=trim($line,"; \t\n\r\0\x0B");
                                                      // add to comments if not empty but only first 3 lines
                                                      if (($s>'') and (count($comment)<3)) $comment[]=$s;
                                                   } else {
                                          
                                                      // non empty line? => clear comments
                                                      if (trim($line)>'') $comment=[];
                                          
                                                   }
                                                }
                                             }
                                          
                                          
                                          ?>
                                          
                                          J 2 Replies Last reply Reply Quote 1
                                          • J
                                            JaredBusch @1337
                                            last edited by

                                            @Pete-S this all hit as I was commuting home.

                                            I have a copy of the files on a server I have access to from home and will be working on this tomorrow.

                                            Thanks.

                                            1 Reply Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 3 / 3
                                            • First post
                                              Last post