Mycroft Community Forum

Best ways to search Mycroft logs

Hey, this started as a reply to @Zerato and @SGee in another thread but it seemed worthwhile as a dedicated topic and I’m keen to hear from others in the community too.

What’s your preferred way to search log files?
Do you use any particular log viewing applications or are you a command line only kind of person?

Mycroft includes a stream of the logs in the CLI view, however I think that is really intended as a real-time display with a few features added in for filtering and searching etc. What if you’re trying to go back and look at your complete log history?

My go to is grep and I thought a few examples might show how grep could be your new best friend (if you do a lot of digging through logs)… :smiley:

  1. Getting all the logs generated by your Skill
  grep "MySkillClass" /var/log/mycroft/skills.log
  1. Get all instances of some error:
  grep "AttributeError" /var/log/mycroft/skills.log

However if you run this you’ll see that it doesn’t give you very useful info - only that the error occured… so let’s fix that

  1. Using the -B (before) and -A (after) flags to get more context about something. For each match of our search pattern, let’s get 10 log lines before and 3 lines after:
  grep -B 10 -A 3 "AttributeError" /var/log/mycroft/skills.log
  1. “But this is showing me lots of errors, some from weeks ago!?!?”. Good point - let’s cut out the older stuff and only show the most recent occurrence:
  grep -B 10 -A 3 "AttributeError" /var/log/mycroft/skills.log | tail -14

Note: Here we want the 10 lines before + the 1 line matching our search pattern + 3 lines after = 14. Also note that the “pipe” character | uses the output from one command as the input to the next command.

  1. If you like to read logs in another viewer, you can also save the output of a grep command into a new file. Here we’ll get a big slab of logs around the error we’re interested in and save the whole thing into a new file that we can open in our favourite log viewer or text editor.
  grep -B 50 -A 30 "AttributeError" /var/log/mycroft/skills.log | tail -81 > attribute-error-log.txt

Note: the > sign writes the output of one command to a file. This will overwrite anything that is already in the file. If you want to keep the existing content and append the new log lines use a double >>


Another few notes:

  • In all the examples above I’ve included quote marks around the pattern to match. This is not required if you are searching for a single word like “AttributeError” however it is needed if you want to search for multiple word phrases like an utterance “what time is it”.
  • grep also lets you use regular expressions and there are lots of resources online about that. If you have any that you think are particularly useful for Mycroft logs please post a comment!
5 Likes

Need a skill to output the relevant log messages to the gui next. :slight_smile:

I’m a command line kind of guy. I might open two terminals side by side and do a ‘tail -f’ on important log files, usually audio.log, voice.log and/or skills.log.

-Mike M
3 Likes

Let’s make this reality. @JarbasAl just wrote an “Application Laucher” and intends to allow alias scripts. We just have to find the appropriate way to stream the output to GUI. Maybe @AIIX can give some thoughts on this.

from gui pow it depends on what format you want the logs displayed on the GUI in for example: classic CLI term style or something more complex like a bunch of specific cards for every different log type it would basically require generating relevant JSON models on the python skill side by parsing the log files which can then easily be displayed in a selected style

1 Like

Simple skill to display and filter logs in GUI

Demo:

Skill: GitHub - AIIX/display-logs-skill: Skill to display and filter logs in Mycroft GUI

5 Likes

That’s amazing. Also, thanks for jumping on what was a silly idea and making it real!

1 Like

not even remotely silly if you count the users having problems to debug mycroft.
Every simplification (even if slightly) is a plus, i guess. (and i would argue against “slightly” here)

Even if the search is only a starting point. But a crucial one.

2 Likes

That looks very damn cool!

Also helpful with the recent addition of configurable log formatting so you can trim down what the log line includes.

1 Like