Follow these steps to maintain more reliable scripts and catch more of your traceback errors:
- automate your scripts to run daily, weekly, monthly, etc.
- log all your traceback errors
- automate aggregating the logs and parsing tracebacks
- start a feedback loop of fixing the tracebacks until 0 tracebacks remain
- re-run the aggregator and confirm tracebacks disappeared
This pure python script allows me to hone in on potential automation problem areas with my scheduled Python scripts.
import os
def parse_tracebacks(log):
errors = list()
with open(log,'r') as f:
for line in f:
if 'Traceback' in line or 'Error' in line:
errors.append([line.strip(),log])
return errors
# parse tracebacks from log files and write to text file
logs = [f for f in os.listdir(os.getcwd()) if '.log' in f.lower()]
tracebacks = [parse_tracebacks(log) for log in logs]
with open('tracebacks.txt', 'w') as fhand:
for t in tracebacks:
for error in t:
fhand.write('%s\n' % ','.join(error))
This script does not catch the entire traceback but shows the error type and log file that contains that error found in a ‘.txt’ and ‘.csv ‘ accessible format. You can save the resulting text file as a csv in a text editor like Notepad.
Benefits
Noteworthy gains from aggregating my logs:
- less fear of missing mistakes
- more freedom to improve the code
- catch the mistakes faster