ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Solved Need to parse large conf files

    IT Discussion
    scripting asterisk config
    6
    53
    3.0k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 1
      1337 @JaredBusch
      last edited by 1337

      @JaredBusch said in Need to parse large conf files:

      @Pete-S said in Need to parse large conf files:

      @JaredBusch
      Also is it just one conf file you need to search through or many in the same directory ?

      many. but feeding that into a loop is simple once I work this out.

      I had file scanning already in the example file but can take it out. The reason I'm asking is if you need to have the filename too in the "report" for each extension?

      JaredBuschJ 1 Reply Last reply Reply Quote 0
      • JaredBuschJ
        JaredBusch @1337
        last edited by

        @Pete-S said in Need to parse large conf files:

        @JaredBusch said in Need to parse large conf files:

        @Pete-S said in Need to parse large conf files:

        @JaredBusch
        Also is it just one conf file you need to search through or many in the same directory ?

        many. but feeding that into a loop is simple once I work this out.

        I had file scanning already in the example file but can take it out. The reason I'm asking is if you need to have the filename too in the "report" for each extension?

        Yes. but why i dunno. this is all a hand coded mess when compared to standards.

        It is all perfectly valid syntax. and there are comments all over. but jsut wow.

        1 Reply Last reply Reply Quote 0
        • matteo nunziatiM
          matteo nunziati
          last edited by matteo nunziati

          ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S

          grep -B3 -n -e "^exten\|^[" $YOURFILENAME | sort -n | uniq | cut -f2 -d: | cut -f2 -d- | grep -e "^[e[\;]"
          

          still not granted to get only valid exten

          JaredBuschJ matteo nunziatiM 2 Replies Last reply Reply Quote 0
          • JaredBuschJ
            JaredBusch @matteo nunziati
            last edited by

            @matteo-nunziati said in Need to parse large conf files:

            ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S

            I'm not waiting. I'm doing my thing too.

            1 Reply Last reply Reply Quote 1
            • matteo nunziatiM
              matteo nunziati @matteo nunziati
              last edited by matteo nunziati

              @matteo-nunziati said in Need to parse large conf files:

              ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S

              grep -B3 -n -e "^exten\|^[" $YOURFILENAME | sort -n | uniq | cut -f2 -d: | cut -f2 -d- | grep -e "^[e[\;]"
              

              still not granted to get only valid exten

              this is the outcome for @Pete-S test file:

              ;; This is a test file
              [context_1]
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              ;;   Ben's inbound 2344242342                      ;
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              exten => 2344242342,1,NoOp()
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              ;;   Bob's inbound 3145551212                      ;
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              exten => 3145551212,1,NoOp()
              ;;  Mary's inbound 4534535345                      ;
              ;   With some added comments                       ;
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              exten => 4534535345,1,NoOp()
              [context_2]
              exten => 33333333,1,NoOp()
              ;    Line 3,,,                                     ;
              ;    Line 4;Let's see if that comment remains      ;
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              exten => 3145454,1,NoOp()
              ;   And lets add this too                         ;;
              ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
              exten => 232342,1,NoOp()
              [context_3]
              exten => 7777777,1
              exten => 8888888,1,NoOp()
              
              JaredBuschJ 1 Reply Last reply Reply Quote 0
              • JaredBuschJ
                JaredBusch @matteo nunziati
                last edited by

                @matteo-nunziati said in Need to parse large conf files:

                ok my last s**t, just in case @JaredBusch doesn't know what to do waiting for @Pete-S
                still not granted to get only valid exten

                doens't work right on a live file. close. but not correct.

                1 Reply Last reply Reply Quote 0
                • JaredBuschJ
                  JaredBusch @JaredBusch
                  last edited by JaredBusch

                  @JaredBusch said in Need to parse large conf files:

                  // find matching files, put in array called filename
                  $filename:=glob("*.ini");

                  it doesn't like the :

                  I moved a copy of the files to the FreePBX 14 box since PHP 5.6 is "current" for CentOS.

                  [jbusch@cccfreepbx newton_conf]$ sudo -u asterisk php parseit.php 
                  PHP Parse error:  syntax error, unexpected ':' in /var/www/html/newton_conf/parseit.php on line 3
                  

                  2821a8fe-1c2b-4c43-9f87-68a5baecf3aa-image.png

                  3817d8e0-37a5-4835-870f-0158ff0cd659-image.png

                  1 2 Replies Last reply Reply Quote 0
                  • 1
                    1337 @JaredBusch
                    last edited by

                    @JaredBusch

                    Hang on a sec. I'm just about finished.

                    1 2 Replies Last reply Reply Quote 0
                    • 1
                      1337 @1337
                      last edited by 1337

                      This is the output:

                      File: sample.conf
                      [context_1]
                      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                      ;;   Bob's inbound 3145551212                      ;
                      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                      exten => 3145551212,1,NoOp()
                      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                      ;;  Mary's inbound 4534535345                      ;
                      ;   With some added comments                       ;
                      exten => 4534535345,1,NoOp()
                      [context_2]
                      exten => 33333333,1,NoOp()
                      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                      ;;   What is this????????????                      ;
                      ;    Lets add a long comment section               ;
                      exten => 3145454,1,NoOp()
                      ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
                      ;;  Let's make a wonky comment line here
                      ;   And lets add this too                         ;;
                      exten => 232342,1,NoOp()
                      [context_3]
                      exten => 7777777,1
                      exten => 8888888,1,NoOp()
                      

                      Personally I'd format it a little for instance by remove all comment lines that are just ;;;;;; and all trailing and preceding ;

                      Or if it's just for viewing I'd html it.

                      1 Reply Last reply Reply Quote 0
                      • 1
                        1337 @1337
                        last edited by

                        This is the complete php file with comments.

                        It takes all *.conf files and goes through them.

                        <?php
                        
                           // find matching files, put in array called filename
                           $filename=glob("*.conf");
                           
                        
                           // scan through all the files
                           foreach ($filename as $f) {
                              // print filename
                              print "File: $f\n";
                        
                              // read all lines into lines array
                              $lines=file($f);
                        
                              $no=0; // linenumber
                              $comment=[]; // comments array
                        
                              // go through the file and pick out what we need
                              foreach ($lines as $line) {
                                 $no++;
                        
                                 // look for [context]
                                 if ($line[0]=='[') {
                                    $context=$line;
                                    print "$context";
                                 }
                                 
                                 // look for extensions
                                 $search="exten =>";
                                 if (substr($line,0,strlen($search))==$search) {
                                    //print $line;
                        
                                    // we found the extension row
                                    // lets take everything after "exten =>"
                                    $extline=trim(substr($line,strlen($search)));
                        
                                    // lets divide it up
                                    $parts=explode(',', $extline);
                                    $ext=$parts[0]; // extension number
                        
                                    // check that it's ?????,1,?????
                                    if ($parts[1]=='1') {
                                       // valid extension, actual extension is $parts[0];
                                       $ext=$line;
                                       // print comments
                                       foreach($comment as $c) print "$c";
                                       // print the extesion and empty line
                                       print "$ext";
                                    }
                                    $comments=[]; // clear comments
                                 }
                                 
                                 // look for comments
                                 if ($line[0]==';') {
                                    // add to comments but only first 3 lines
                                    if (count($comment)<3) $comment[]=$line;
                                    //print_r($comment);
                                 } else {
                        
                                    // non empty line? => clear comments
                                    if (trim($line)>'') $comment=[];
                        
                                 }
                              }
                           }
                        
                        ?>
                        
                        1 Reply Last reply Reply Quote 0
                        • 1
                          1337 @JaredBusch
                          last edited by 1337

                          @JaredBusch said in Need to parse large conf files:

                          @JaredBusch said in Need to parse large conf files:

                          // find matching files, put in array called filename
                          $filename:=glob("*.ini");

                          it doesn't like the :

                          I moved a copy of the files to the FreePBX 14 box since PHP 5.6 is "current" for CentOS.

                          It's because I've been working with another language all day that uses := to assign a variable and not just =

                          Takes me a few minutes to reboot my brain into php mode. :smiling_face_with_smiling_eyes:

                          1 1 Reply Last reply Reply Quote 0
                          • 1
                            1337 @1337
                            last edited by 1337

                            Just for fun I found some extensions.conf file on the net to try. It led me to add some better formatting so the output is more readable.

                            PS. Just a small change like this that would have caused hours of search on regex, options for grep, awk, sed etc to try and make something work. That's why I think it's easier to just go full on programming language as soon as it is more complex than just finding a few lines that matches.

                            I know as this is all for naught but whatever.

                            Output from the php script:

                            File: sample.conf
                            
                            [context_1]
                            ; Bob's inbound 3145551212
                            exten => 3145551212,1,NoOp()
                            
                            ; Mary's inbound 4534535345
                            ; With some added comments
                            exten => 4534535345,1,NoOp()
                            
                            [context_2]
                            exten => 33333333,1,NoOp()
                            
                            ; What is this????????????
                            ; Lets add a long comment section
                            ; Line 3,,,
                            exten => 3145454,1,NoOp()
                            
                            ; Let's make a wonky comment line here
                            ; And lets add this too
                            exten => 232342,1,NoOp()
                            
                            [context_3]
                            exten => 7777777,1
                            
                            exten => 8888888,1,NoOp()
                            

                            PHP script v2

                            <?php
                               // Ver 2 - improved formatting, only showing headings that have extensions
                               
                               // find matching files, put in array called filename
                               $filename=glob("*.conf");
                               
                            
                               // scan through all the files
                               foreach ($filename as $f) {
                                  // print filename
                                  print "File: $f\n\n";
                            
                                  // read all lines into lines array
                                  $lines=file($f);
                            
                                  $no=0; // linenumber
                                  $comment=[]; // comments array
                                  $context=''; // context text
                            
                                  // go through the file and pick out what we need
                                  foreach ($lines as $line) {
                                     $no++;
                            
                                     // look for [context]
                                     if ($line[0]=='[') {
                                        $context=trim($line);
                                     }
                                     
                                     // look for extensions
                                     $search="exten =>";
                                     if (substr($line,0,strlen($search))==$search) {
                                        //print $line;
                            
                                        // we found the extension row
                                        // lets take everything after "exten =>"
                                        $extline=trim(substr($line,strlen($search)));
                            
                                        // lets divide it up
                                        $parts=explode(',', $extline);
                            
                                        // check that it's ?????,1,?????
                                        if ($parts[1]=='1') {
                                           // valid extension, actual extension is $parts[0];
                                           $ext=trim($line);
                                           
                                           // print context if it has not been printed
                                           if ($context>'') { print "$context\n"; $context=''; }
                                           // print comments
                                           foreach($comment as $c) print "; $c\n";
                                           // print the extension and empty line
                                           print "$ext\n\n";
                                        }
                                        $comments=[]; // clear comments
                                     }
                                     
                                     // look for comments
                                     if ($line[0]==';') {
                                        // remove whitespace and ;
                                        $s=trim($line,"; \t\n\r\0\x0B");
                                        // add to comments if not empty but only first 3 lines
                                        if (($s>'') and (count($comment)<3)) $comment[]=$s;
                                     } else {
                            
                                        // non empty line? => clear comments
                                        if (trim($line)>'') $comment=[];
                            
                                     }
                                  }
                               }
                            
                            
                            ?>
                            
                            JaredBuschJ 2 Replies Last reply Reply Quote 1
                            • JaredBuschJ
                              JaredBusch @1337
                              last edited by

                              @Pete-S this all hit as I was commuting home.

                              I have a copy of the files on a server I have access to from home and will be working on this tomorrow.

                              Thanks.

                              1 Reply Last reply Reply Quote 0
                              • JaredBuschJ
                                JaredBusch @1337
                                last edited by

                                @Pete-S Well tomorrow becamse the following Wednesday.. But. this works perfectly.

                                Now to figure out WTF all this stuff is that I need to replicate into FreePBX.

                                8c904acf-bfbc-480e-855a-b0985bd7c306-image.png

                                073eba70-ab8c-4c70-b633-2b41ea9972a2-image.png

                                1 1 Reply Last reply Reply Quote 1
                                • 1
                                  1337 @JaredBusch
                                  last edited by

                                  @JaredBusch Glad it worked as intended.

                                  1 Reply Last reply Reply Quote 0
                                  • 1
                                  • 2
                                  • 3
                                  • 3 / 3
                                  • First post
                                    Last post