Faculty of Engineering and Mathematical Sciences 
Not logged in (login)

help4407


This forum is provided to promote discussion amongst students enrolled in Open Source Tools and Scripting (CITS4407).

Assignment 2 and Clarifications.
 
Options:
RSS cloud
Jump to:

Assign 2 Q2 - execution time very long

3 of 390 articles shown, currently no other people reading this forum.
photo
From: Fraser L.
Date: Tue 19th May, 11:56am
Actions: 
        Login-to-reply
Currently my code takes a very long time to process through all of secure-access.log-
20200510 (like 45 minutes on my laptop). Are there any marks or penalties for slow 
executing code? For ref in my while loop which iterates through every line i so far 
have an awk command, a reg exp check (in an if statement) and an array append. 

Assign 2 Q2 - execution time very long

photo
From: Christopher M.
Date: Tue 19th May, 12:12pm
Actions: 
        Login-to-reply
"Fraser Loneragan"                               wrote:

> Currently my code takes a very long time to process through all of secure-access.log-
> 20200510 (like 45 minutes on my laptop). Are there any marks or penalties for slow 
> executing code? For ref in my while loop which iterates through every line i so far 
> have an awk command, a reg exp check (in an if statement) and an array append. 

Hi Fraser,

45 minutes is clearly far too long (but I admire your patience!).
Invoking new processes takes the most time and, I suspect, that you're invoking awk for every 
line of the log file 93000x).

Can you restructure your approach, so that you invoke external programs, such as awk, far fewer 
times (maybe just once).  Feed the input file to awk, have it read each line, perform the RE 
matches on each line, and leaving its results, even temporary results in external files, rather 
than in shell variables.

Possible?

Assign 2 Q2 - execution time very long

photo
From: David M.
Date: Tue 19th May, 3:22pm
Actions: 
        Login-to-reply
Things I have found helpful for script debugging and performance measurement,
in decreasing order of importance are:

1. Print debug statements in the code at suitable places.

2. Set bash debugging option -x to confirm the script is working as you intended.

3. Use the "time" command to get a general idea of where the computer is spending time.

4. Use the "gprof" and "strace" commands for performance analysis (rarely used).
This Page


Program written by: [email protected]
Feedback welcome
Last modified:  8:27am May 24 2020