ottyl/README.md

# ottyl

Observability dashboard from terminal output

# Usage

At a high level, the idea is that a program will output logs, which can be captured and displayed on a timeline and dashboard similar to grafana.

```bash
cli_command --verbose | ottyl

```

# Architecture

ottyl reads the logs via stdin and indexes it in-memory in a BTreeMap for property search.

# Engine

This is the main program, hosting the webapp over axum.

The engine reads stdin and sends line-split messages to a log ingester.
The log ingester holds a mutex over a BTreeMap<IndexKey, DataValue>.

The BTreeMap may be a custom variation of the std library one, because we need to be able to scan specific ranges and jump to keys.
```
/// Get the given key, or the next key in the btree
fn get_or_next(&self, key: &K) -> Option<Entry<K, V>>{...} 

/// Get the next key in the btree
fn get_or_next(&self, key: &K) -> Option<Entry<K, V>>{...} 

/// Get the previous key in the btree
fn previous(&self, key: &K) -> Option<Entry<K, V>> {...}
```

IndexKey is how we will index the logs and metrics required for search and dashboards.
```
enum IndexKey {
  // Convenient way to find the start of the range
  // It should never be inserted into the btreemap, but it can be used for navigating it
  // This is true for all position-marker type keys
  Start,
  // A key for a literal raw log, where sequence is effectively the line number
  // and timestamp is the time at which it was ingested into ottyl
  IngestedLog{sequence: usize, timestamp: Instant},
  // Convenience to get the end of the ingested log range and start of param range
  LogParamSplit,
  // Param declarations allow the user to specific how a parameter for a message can be derived
  // This is useful for deriving timestamps from logs, that are different from the ingested timestamp (ex cat log_dump.txt)
  // Other metrics may also be derived, like log level etc
  ParamDeclaration{name: String},
  // End of param range, start of metric declaration range
  ParamMetricSplit,
  // Used for storing the exact metric definition, so buckets and parameters can be derived
  MetricDeclaration{name: String},
  // Used to split metric declarations from their buckets
  MetricBucketSplit,
  // 
  Bucket{metric_name: String, labels: BTreeMap<String, MetricPrimitive>}
  // End of key range, nothing should ever come after this
  End,
}

```

DataValue is how we have a convenient pairing for a single BTreeMap KV mapping.
It covers all the result types for the primary BTreeMap.

```
enum DataValue {
  // Stores the raw ingested log
  // Used in IngestedLog keys
  LogLiteral{text: String},
  // Stores a metric declaration
  // Used in MetricDeclaration keys
  MetricDeclaration{message_match_regex: String, params: Vec<RegexToMetricPrimitiveMatcher>
  // Stores the values of a bucket
  // Used in Bucket keys
  BucketValue{values: Vec<(Sequence, Timestamp, MetricPrimitive)>}
}
```

MetricPrimitive is literally just a simple way to track values without using json.
```
enum MetricPrimitive {
  String(String),
  Number(Number),
  Timestamp(Instant),
}
```


# WebApp

ottyl provides a spa webapp dashboard in typescript react and tailwind.
It is served from an in-process axum binding.

```mermaid
block
columns 2
  NavMenu
  block
  columns 1
    FilterBar
    LogTimeline
    LogPreview
    Metrics
    Dashboards
  end
```

## NavMenu

The nav menu is largely for managing settings and presets.

Presets track how to parse and index logs, and stored dashboards.
That means that restarting the application for an CLI app does not lose dashboards and parsing rules.

Settings involve colour schemes, or declaring the path for the css.
It should be possible to import a css, so it gets stored in the same location that the profiles and settings are stored.
It should be possible to delete imported css.
Not all css needs to be imported, linking to some path or url should be viable also - those are not deletable.

Settings "owns" the profiles, so changing a settings file changes the used profiles.
At runtime, there is only 1 settings.
It can be set using clap cli args.

The settings file by default is stored in a sensible user location for cli settings.

## FilterBar

Syntax aware text search.
A parser can be written in pest.
For now, the syntax should support regex searching and metrics searching, such as `metric_name[param]<5`.
LogSequence ('sequence') and Timestamp ('timestamp') are reserved metrics.

```
regex("error") and sequence<5 or timestamp>12:00
```

DateTime parsing should support several formats
- relative humantime (`5h`, `-3m` etc)
- absolute
- fulldate



## LogTimeline

This should work similar to how elasticsearch has a log timeline.
Its a timeline of a barchart, where each bar is effectively some bucket of messages.
It reflects the selected time range.
Dragging over the log timeline displays a rectangle as it is happening and also the time range.
Once the drag stops, the selected range becomes the new filter time range.

## LogPreview

Log preview is simply a scrollable table of the log messages, like in kibana.
The sequence, and timestamp are visible.
It should be possible to pick which parameters are visible.

## Metrics

Metrics can be declared from logs.
There should be a language syntax for declaring metircs.

Once declared, already-ingested logs get indexed on the metric, and new metrics get ingested as well.

## Dashboards

Dashboards work the same way as in grafana.
There should be support for promql queries to derive the visualisation.
Supported charts should be bar chart, line chart, and pie chart.