• Monitoring BBS logs

    From Dan Clough@1:135/115 to All on Thu Jan 25 21:09:00 2024
    Hello all,

    Wondering what you all do/use for monitoring logs on your boards.

    I have a few terminals open to "tail" various logs on my system (running Synchronet). Been doing it that way for a long while, and it works fine,
    I guess.

    Has anyone used the 'lnav' utility to watch logs? It's pretty slick, has
    LOTS of options, and I like the use of colors in logs to help things "jump
    out" at you. If anyone is using this, and has taken the time to customize
    the look/feel of how it displays, would you comment on how you're using
    it, and would you maybe be willing to share your custom config that I
    could use as a starting point to learn how it's done?

    Appreciate any input you may have!


    ... No one is listening until you make a mistake.
    === MultiMail/Linux v0.52
    --- SBBSecho 3.20-Linux
    * Origin: Palantir * palantirbbs.ddns.net * Pensacola, FL * (1:135/115)
  • From Mike Miller@1:154/30 to Dan Clough on Thu Feb 1 14:26:12 2024

    Hello Dan!

    25 Jan 24 21:09, you wrote to all:

    Hello all,

    Wondering what you all do/use for monitoring logs on your boards.

    I have a few terminals open to "tail" various logs on my system
    (running Synchronet). Been doing it that way for a long while, and it works fine, I guess.

    Has anyone used the 'lnav' utility to watch logs? It's pretty slick,
    has LOTS of options, and I like the use of colors in logs to help
    things "jump out" at you. If anyone is using this, and has taken the
    time to customize the look/feel of how it displays, would you comment
    on how you're using it, and would you maybe be willing to share your custom config that I could use as a starting point to learn how it's
    done?

    Appreciate any input you may have!


    The free grafana-cloud tier is plenty for monitoring my bbs server. I'm streaming the logs to it via the grafana-agent.

    Last week I got a wild hair and set up this:

    https://snapshots.raintank.io/dashboard/snapshot/ybCmnOlQAJnFye872Dsmdi1ev7EV038q

    (snapshot from the last 24 hours as of this post)


    I still plan on adding number of messages from/to uplinks/downlinks. I may actually send out alerts if dupes rise above a certain pecentage.



    I do have another dashboard that I'm working on. It has number of connections over time, number of connections by IP address, and a few other things.


    doing this via logs is not all that efficient, though. At some point I'll likely create some kind of prometheus exporter to just scrape data, rather than forcing grafana to parse its entire history of logs (only 48 hours worth on the free tier) to display data.

    Grafana will, however, allow you to set up alerts based on triggers, such as a specific line in a log file. SBBS choked on me in a weird way the other day, for the first time in forever, so I may set it up to send an alert to my phone if something like that happens again. I didn't catch it, and the board was down for more than a day.





    Mike


    ... A blind gynecologist can read lips.
    === GoldED+/LNX 1.1.5-b20231112
    --- SBBSecho 3.20-Linux
    * Origin: War Ensemble - warensemble.com - Appleton, WI (1:154/30)
  • From Rob Swindell@1:103/705 to Mike Miller on Thu Feb 1 13:03:33 2024
    Re: Monitoring BBS logs
    By: Mike Miller to Dan Clough on Thu Feb 01 2024 02:26 pm

    I do have another dashboard that I'm working on. It has number of connections over time, number of connections by IP address, and a few other things.


    doing this via logs is not all that efficient, though. At some point I'll likely create some kind of prometheus exporter to just scrape data, rather than forcing grafana to parse its entire history of logs (only 48 hours worth on the free tier) to display data.

    Grafana will, however, allow you to set up alerts based on triggers, such as a specific line in a log file. SBBS choked on me in a weird way the other day, for the first time in forever, so I may set it up to send an alert to my phone if something like that happens again. I didn't catch it, and the board was down for more than a day.

    This sounds like a good use for MQTT:
    https://wiki.synchro.net/ref:mqtt
    --
    digital man (rob)

    This Is Spinal Tap quote #35:
    Jeanine Pettibone: You don't do heavy metal in Dubly, you know.
    Norco, CA WX: 48.7øF, 93.0% humidity, 3 mph NNW wind, 0.77 inches rain/24hrs --- SBBSecho 3.20-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Dan Clough@1:135/115 to Mike Miller on Thu Feb 1 20:04:00 2024
    Mike Miller wrote to Dan Clough <=-

    Hello all,

    Wondering what you all do/use for monitoring logs on your boards.

    I have a few terminals open to "tail" various logs on my system
    (running Synchronet). Been doing it that way for a long while, and it works fine, I guess.

    Has anyone used the 'lnav' utility to watch logs? It's pretty slick,
    has LOTS of options, and I like the use of colors in logs to help
    things "jump out" at you. If anyone is using this, and has taken the
    time to customize the look/feel of how it displays, would you comment
    on how you're using it, and would you maybe be willing to share your custom config that I could use as a starting point to learn how it's
    done?

    Appreciate any input you may have!


    The free grafana-cloud tier is plenty for monitoring my bbs
    server. I'm streaming the logs to it via the grafana-agent.

    Last week I got a wild hair and set up this:

    https://snapshots.raintank.io/dashboard/snapshot/ybCmnOlQAJnFye872 Dsmdi1ev7EV038q

    (snapshot from the last 24 hours as of this post)

    Wow, that's pretty awesome!

    I still plan on adding number of messages from/to
    uplinks/downlinks. I may actually send out alerts if dupes rise
    above a certain pecentage.

    I do have another dashboard that I'm working on. It has number of connections over time, number of connections by IP address, and a
    few other things.

    doing this via logs is not all that efficient, though. At some
    point I'll likely create some kind of prometheus exporter to just
    scrape data, rather than forcing grafana to parse its entire
    history of logs (only 48 hours worth on the free tier) to display
    data.

    I have no knowledge of this kind of thing, seems above my comfort level
    for sure. The free tier does seem to be fairly useful, if one can grasp
    the overall big-picture of what it's doing. I may create an account and
    try to figure some of it out. Thanks for pointing this out.

    Grafana will, however, allow you to set up alerts based on
    triggers, such as a specific line in a log file. SBBS choked on
    me in a weird way the other day, for the first time in forever,
    so I may set it up to send an alert to my phone if something like
    that happens again. I didn't catch it, and the board was down
    for more than a day.

    That sounds like a great feature too. Thanks again for your reply and
    this information.

    Dan



    ... System halted - Press all keys at once to continue.
    === MultiMail/Linux v0.52
    --- SBBSecho 3.20-Linux
    * Origin: Palantir * palantirbbs.ddns.net * Pensacola, FL * (1:135/115)