logpare

Quick Start

Compress your first log file from the CLI and from TypeScript, and read the output formats.

This guide will get you up and running with logpare in minutes.

Basic CLI Usage

Compress a log file:

logpare server.log

Use stdin:

cat /var/log/syslog | logpare
tail -1000 app.log | logpare

Get detailed output:

logpare --format detailed error.log

Output as JSON:

logpare --format json --output compressed.json access.log

There is no compress subcommand. logpare takes options and file paths directly — logpare compress app.log would look for a file named compress and exit with an error.

Basic Programmatic Usage

Compressing an Array of Lines

import { compress } from 'logpare';

const logs = [
  'INFO Connection from 192.168.1.1 established',
  'INFO Connection from 192.168.1.2 established',
  'ERROR Connection timeout after 30s',
  'INFO Connection from 10.0.0.1 established',
];

const result = compress(logs);

console.log(result.formatted);
// === Log Compression Summary ===
// Input: 4 lines → 2 templates (50.0% reduction)
//
// Top templates by frequency:
// 1. [3x] INFO Connection from <*> established
// 2. [1x] ERROR Connection timeout after <*>

console.log(result.stats);
// {
//   inputLines: 4,
//   uniqueTemplates: 2,
//   compressionRatio: 0.5,
//   estimatedTokenReduction: 0.225,
//   droppedLines: 0,
//   processingTimeMs: 1
// }

compressionRatio and estimatedTokenReduction are ratios between 0 and 1, not percentages. Multiply by 100 yourself if you want to print a percentage.

Compressing Text

import { compressText } from 'logpare';
import { readFileSync } from 'node:fs';

const logFile = readFileSync('app.log', 'utf-8');
const result = compressText(logFile);

console.log(result.formatted);

Getting Structured Data

result.templates is always available, whatever the format — format only controls the formatted string.

const result = compress(logs);

result.templates.forEach(template => {
  console.log(`Pattern: ${template.pattern}`);
  console.log(`Occurrences: ${template.occurrences}`);
  console.log(`Severity: ${template.severity}`);
  console.log(`URLs: ${template.urlSamples.join(', ')}`);
  console.log('---');
});

Passing Options

compress() takes format and maxTemplates at the top level. Every Drain algorithm parameter is nested under drain:

const result = compress(logs, {
  format: 'detailed',
  maxTemplates: 20,
  drain: {
    depth: 5,
    simThreshold: 0.5,
  },
});

Passing { depth: 5 } at the top level is a common mistake — it is silently ignored and does not typecheck. Drain options always go inside drain.

Output Formats

Summary (Default)

Compact overview with top templates and rare events:

const result = compress(logs, { format: 'summary' });

Output:

=== Log Compression Summary ===
Input: 10,847 lines → 23 templates (99.8% reduction)

Top templates by frequency:
1. [4,521x] INFO Connection from <*> established
2. [3,892x] DEBUG Request <*> processed in <*>

Rare events (≤5 occurrences): 1 templates
- [1x] FATAL Database connection lost

Detailed

Full information with all diagnostic metadata:

const result = compress(logs, { format: 'detailed' });

Output:

=== Log Compression Details ===
Input: 10,847 lines → 23 templates (99.8% reduction)
Estimated token reduction: 95.0%

=== Template t001 (4,521 occurrences) ===
Pattern: INFO Connection from <*> established
Severity: info
First seen: line 1
Last seen: line 10,234
URLs:
  - https://api.example.com/v1/users
Status codes: 200, 201
Durations: 45ms, 120ms
Sample variables:
  - 192.168.1.1
  - 10.0.0.55

JSON

Machine-readable format:

const result = compress(logs, { format: 'json' });

Output:

{
  "version": "1.1",
  "stats": {
    "inputLines": 10847,
    "uniqueTemplates": 23,
    "compressionRatio": 0.998,
    "estimatedTokenReduction": 0.95
  },
  "templates": [
    {
      "id": "t001",
      "pattern": "INFO Connection from <*> established",
      "occurrences": 4521,
      "severity": "info",
      "isStackFrame": false,
      "samples": [["192.168.1.1"], ["10.0.0.55"]],
      "urlSamples": [],
      "fullUrlSamples": [],
      "statusCodeSamples": [],
      "correlationIdSamples": [],
      "durationSamples": [],
      "firstSeen": 0,
      "lastSeen": 10233
    }
  ]
}

Note that in JSON output the field is samples (not sampleVariables), and firstSeen/lastSeen are zero-based line indices. The detailed formatter displays them as one-based line numbers.

JSON Stable

Same data with recursively sorted keys and no whitespace, so repeated compressions of the same logs serialize byte-identically — useful for diffing and for LLM prompt caching:

const result = compress(logs, { format: 'json-stable' });

Advanced: Incremental Processing

For streaming or very large log files, use the Drain API directly:

import { createDrain } from 'logpare';

const drain = createDrain({
  depth: 4,
  simThreshold: 0.4,
});

// Process logs one at a time
drain.addLogLine('ERROR Connection failed');
drain.addLogLine('ERROR Connection timeout');
drain.addLogLine('INFO Request completed');

// Or a batch at once
drain.addLogLines(['INFO Request completed', 'INFO Request completed']);

// Read templates at any point
for (const template of drain.getTemplates()) {
  console.log(`[${template.occurrences}x] ${template.pattern}`);
}

// Or get a formatted result
console.log(drain.getResult('summary').formatted);

Note that createDrain() takes Drain options flat — it receives DrainOptions directly, unlike compress(), which nests them under drain.

Keep your original logs

Compression is a diagnostic representation, not an archive format. Templates collapse repeated lines into a pattern plus counts and sampled values, so the original text cannot be reconstructed from the output. Keep the raw logs; send the compressed view to the model.

What's Next?

Now that you've learned the basics:

Installation

Install logpare globally, per-project, or run it with npx, and confirm the install works.

compress()

Compress an array of log lines into semantic templates — the main entry point for most callers.

On this page

Basic CLI UsageBasic Programmatic UsageCompressing an Array of LinesCompressing TextGetting Structured DataPassing OptionsOutput FormatsSummary (Default)DetailedJSONJSON StableAdvanced: Incremental ProcessingKeep your original logsWhat's Next?