diff --git a/README.md b/README.md index 885f5c3..63bdb33 100644 --- a/README.md +++ b/README.md @@ -1,226 +1,52 @@ # ottyl -Observability dashboard from terminal output - -# Usage - -At a high level, the idea is that a program will output logs, which can be captured and displayed on a timeline and dashboard similar to grafana. +Observability for any command's terminal output — no logging backend, no agents, no config to ship. ```bash -cli_command --verbose | ottyl - +your_app --verbose | ottyl ``` -# Building +Pipe a program's stdout into `ottyl` and turn it into a searchable log timeline, ad-hoc metrics extracted from the log lines themselves, and Grafana-style dashboards — served from a single local binary, for the lifetime of that process: -The webapp is embedded into the `ottyl` binary at compile time, so `webapp/dist` must exist *before* running `cargo build`: +- **Search:** A syntax-aware filter bar over your logs — regex matches and metric comparisons like `latency_ms[value]<200`, with relative (`5h`, `-3m`) and absolute datetimes. +- **Log timeline:** A bucketed bar chart of message volume over time, Kibana-style. Drag across it to set the active time range. +- **Metrics from logs:** Declare a metric by matching a regex against your log lines and extracting a value. It applies retroactively to everything already ingested, and to everything that follows. +- **Dashboards:** Bar, line, and pie chart panels driven by PromQL-style queries over your declared metrics. +- **Mixed sources:** Poll an existing Prometheus/OpenMetrics endpoint alongside log ingestion, in the same dashboard. +- **Profiles:** Parsing rules and dashboards are saved per app, so they're there again next time that app runs through ottyl. + +## Use cases + +- Debugging a noisy CLI or long-running script, instead of scrolling a terminal buffer. +- Getting metrics and charts out of a program that only ever prints text. +- Local dashboards without standing up a metrics stack. +- Combining scraped Prometheus metrics with log-derived ones during a debugging session. + +## Get started + +Requires Rust and Node.js. The webapp is built and embedded into the binary, so the result is a single self-contained executable. ```bash -cd webapp && npm install && npm run build && cd .. -cargo build --release +git clone +cd ottyl +make build +your_app --verbose | ottyl ``` -The resulting binary is self-contained — no `webapp/dist` directory or `--static-dir` flag is needed at runtime. - -# Architecture - -ottyl reads the logs via stdin and indexes it in-memory in a BTreeMap for property search. - -# Engine - -This is the main program, hosting the webapp over axum. - -The engine reads stdin and sends line-split messages to a log ingester. -The log ingester holds a BTreeMap. - -IndexKey is how we will index the logs and metrics required for search and dashboards. -``` -enum IndexKey { - // Convenient way to find the start of the range - // It should never be inserted into the btreemap, but it can be used for navigating it - // This is true for all position-marker type keys - Start, - // A key for a literal raw log, where sequence is effectively the line number - // and timestamp is the time at which it was ingested into ottyl - IngestedLog{sequence: usize, timestamp: Instant}, - // Convenience to get the end of the ingested log range and start of param range - LogParamSplit, - // Param declarations allow the user to specific how a parameter for a message can be derived - // This is useful for deriving timestamps from logs, that are different from the ingested timestamp (ex cat log_dump.txt) - // Other metrics may also be derived, like log level etc - ParamDeclaration{name: String}, - // End of param range, start of metric declaration range - ParamMetricSplit, - // Used for storing the exact metric definition, so buckets and parameters can be derived - MetricDeclaration{name: String}, - // Used to split metric declarations from their buckets - MetricBucketSplit, - // - Bucket{metric_name: String, labels: BTreeMap} - // End of key range, nothing should ever come after this - End, -} +Or point it at a log file instead of stdin, and mix in a Prometheus/OpenMetrics endpoint: +```bash +ottyl --file /var/log/your_app.log --metrics-url http://localhost:9090/metrics ``` -DataValue is how we have a convenient pairing for a single BTreeMap KV mapping. -It covers all the result types for the primary BTreeMap. +ottyl prints the URL of the local web server it starts (an OS-assigned free port by default; pin one with `--port`). See `ottyl --help` for the rest, including where settings and profiles live (`--settings-path`). +Full documentation — filtering syntax, parameters, metrics, dashboards — is built into the app itself, under the nav menu's docs section. + +## Development + +```bash +make dev # frontend dev server with hot reload, proxying to a separately running `cargo run` backend +make test # backend + frontend tests +make lint # clippy + oxlint ``` -enum DataValue { - // Stores the raw ingested log - // Used in IngestedLog keys - LogLiteral{text: String}, - // Stores a metric declaration - // Used in MetricDeclaration keys - MetricDeclaration{message_match_regex: String, params: Vec - // Stores the values of a bucket - // Used in Bucket keys - BucketValue{values: Vec<(Sequence, Timestamp, MetricPrimitive)>} -} -``` - -MetricPrimitive is literally just a simple way to track values without using json. -``` -enum MetricPrimitive { - String(String), - Number(Number), - Timestamp(Instant), -} -``` - -## Tasks - -The ingestion tasks reads stdin, sequences the logs, timestamps them and sends over a channel to the engine task. - -The engine task processes incoming logs and incoming requests. -It should be lock-free and the sole-owner of the btreemap. - -Axum is not a specific task, but exists and serves the API and frontend. -API requests get validated (like, correct syntax etc etc) and then requests are sent over a channel to the engine. -The request includes a byte buffer, for response serialisation to go into it. -The engine does the serialisation, because it is the sole owner of the btreemap and should not be copying memory simply to serialise it later. -A callback is given, so the API knows the request is processed and response is ready to send back from the byte buffer. -Rkyv can be used if that helps, otherwise sending raw bytes is fine as well. - -# WebApp - -ottyl provides a spa webapp dashboard in typescript react and tailwind. -It is served from an in-process axum binding. - -```mermaid -block -columns 2 - NavMenu - block - columns 1 - FilterBar - LogTimeline - LogPreview - Metrics - Dashboards - end -``` - -## NavMenu - -The nav menu is largely for managing settings and presets. - -Presets track how to parse and index logs, and stored dashboards. -That means that restarting the application for an CLI app does not lose dashboards and parsing rules. - -Settings involve colour schemes, or declaring the path for the css. -It should be possible to import a css, so it gets stored in the same location that the profiles and settings are stored. -It should be possible to delete imported css. -Not all css needs to be imported, linking to some path or url should be viable also - those are not deletable. - -Settings "owns" the profiles, so changing a settings file changes the used profiles. -At runtime, there is only 1 settings. -It can be set using clap cli args. - -The settings file by default is stored in a sensible user location for cli settings. - -The Profiles are basically a dropdown. -There is always a "Default" profile. -The dropdown should have a an option to create a new profile - the user needs to provide a name, otherwise its a copy of the current profile. -Profiles can be deleted; Default profile cannot be deleted. - -Docs should be included as well. -The navbar for docs should be collapsing accordion style for the various sections. -There should be the following pages -- General idea (ingestion, profiles, settings, CLI args) -- Filtering -- Parameters -- Metrics -- Dashboards - -## FilterBar - -Syntax aware text search. -A parser can be written in pest. -For now, the syntax should support regex searching and metrics searching, such as `metric_name[param]<5`. -LogSequence ('sequence') and Timestamp ('timestamp') are reserved metrics. - -``` -regex("error") and sequence<5 or timestamp>12:00 -``` - -DateTime parsing should support several formats -- relative humantime (`5h`, `-3m` etc) -- absolute -- fulldate - - - -## LogTimeline - -This should work similar to how elasticsearch has a log timeline. -Its a timeline of a barchart, where each bar is effectively some bucket of messages. -It reflects the selected time range. -Dragging over the log timeline displays a rectangle as it is happening and also the time range. -Once the drag stops, the selected range becomes the new filter time range. - -## LogPreview - -Log preview is simply a scrollable table of the log messages, like in kibana. -The sequence, and timestamp are visible. -It should be possible to pick which parameters are visible. - -## Metrics - -Metrics can be declared from logs. -There should be a language syntax for declaring metircs. - -Once declared, already-ingested logs get indexed on the metric, and new metrics get ingested as well. - -## Dashboards - -Dashboards work the same way as in grafana. -There should be support for promql queries to derive the visualisation. -Supported charts should be bar chart, line chart, and pie chart. - - -# Ideas - -This part describes ideas that can be implemented in the future, but not now. - -## Ntfy.sh alerts - -Have matchers you are interested in. -Configure the ntfy.sh channel. -Send to the channel when an event triggers. - -## Sequence diagram of events - -It would be awesome if it were possible to declare participants of a sequence diagram using parameters or matchers. -Example `node_id` for a cluster test or something. -Then it may be possible to capture specific events of interest. -There wouldn't be a relation between nodes, only the sequence of events determines relations in the sequence diagram. - -## Index allocation size - -It should be possible to see how much ram is being consumed by the program. -What should happen if it reaches a limit? - - -