Class CsvDumper

Asynchronously appends numeric samples to a CSV file using a bounded, non-blocking queue.

Inheritance
CsvDumper
Implements
Namespace: MOSAIC.Components.Basics
Assembly: MOSAIC.dll
Syntax
public class CsvDumper : IAsyncDisposable
Remarks

CsvDumper is designed for real-time producers (e.g., sensors, signal-processing blocks) that must not block on disk I/O. Calls to Enqueue(double, double) and its overloads attempt to write a stable numeric row snapshot into an internal Channel<T>.

Automatic logging: When a CsvDumper is assigned to Dumper, every successful Publish(object) call automatically logs the published value via Enqueue(double, object). This eliminates per-block boilerplate. Blocks no longer need to manage their own dumper fields, enqueue calls, or disposal.

Supported types for Enqueue(double, object):

  • MathNet.Numerics.LinearAlgebra.Vector<T> of double — logged as timestamp, v₀, v₁, …
  • double[] — logged as timestamp, v₀, v₁, …
  • double — logged as timestamp, value
  • ValueTuple<string, Vector<double>> — logged as timestamp, label, v₀, v₁, …
  • ValueTuple<string, double[]> — logged as timestamp, label, v₀, v₁, …
  • MathNet.Numerics.LinearAlgebra.Matrix<T> of double — logged as one row per matrix row (future-ready)
  • IEnumerable<T> of double — logged as timestamp, v₀, v₁, …

Unsupported types are silently skipped.

Backpressure behavior: The internal channel is bounded (8 192 slots). When full, new rows are rejected and counted in DroppedRows (Wait).

Flush behavior: The writer flushes periodically (default: 10 s) and always performs a final flush during disposal.

Threading: Multiple producers are supported (SingleWriter = false). A single background consumer task performs the file writes (SingleReader = true).

Culture: All numeric values use InvariantCulture to ensure a stable decimal separator (.) regardless of system locale.

Constructors

CsvDumper(string, string, TimeSpan?, int)

Initializes a new CSV dumper that appends rows to a file on a background writer task.

Declaration
public CsvDumper(string filePath, string fileName = "out.csv", TimeSpan? flushEvery = null, int queueCapacity = 8192)
Parameters
Type Name Description
string filePath

Directory path where the CSV file will be created.

string fileName

Base file name (without extension). The implementation appends .csv.

TimeSpan? flushEvery

Flush interval for the underlying StreamWriter. If null, defaults to 10 seconds.

int queueCapacity
Remarks

The file is opened in append mode (Append). The target directory is created if it does not already exist.

The background writer task is started immediately upon construction. Dispose the instance via DisposeAsync() to guarantee all queued rows are flushed.

Fields

_channel

Bounded channel used as a non-blocking queue between producer threads and the background writer.

Declaration
private readonly Channel<CsvDumper.CsvCommand> _channel
Field Value
Type Description
Channel<CsvDumper.CsvCommand>

_droppedRows

Number of rejected data rows; flush commands wait for space and are never dropped.

Declaration
private long _droppedRows
Field Value
Type Description
long

_filePath

Full path (including file name) of the output CSV file.

Declaration
private readonly string _filePath
Field Value
Type Description
string

_flushEvery

Interval between periodic flushes of the underlying StreamWriter.

Declaration
private readonly TimeSpan _flushEvery
Field Value
Type Description
TimeSpan

_lastError

Declaration
private Exception? _lastError
Field Value
Type Description
Exception

_writerTask

Handle to the background writer task spawned in the constructor.

Declaration
private readonly Task _writerTask
Field Value
Type Description
Task

Properties

DroppedRows

Declaration
public long DroppedRows { get; }
Property Value
Type Description
long

LastError

Declaration
public Exception? LastError { get; }
Property Value
Type Description
Exception

QueuedCommands

Declaration
public int QueuedCommands { get; }
Property Value
Type Description
int

Methods

AppendDouble(StringBuilder, double)

Appends a culture-invariant, high-precision representation of a double to a string builder. Uses G17 for full IEEE 754 round-trip precision.

Declaration
private static void AppendDouble(StringBuilder sb, double d)
Parameters
Type Name Description
StringBuilder sb
double d

AppendRow(StringBuilder, CsvCommand)

Declaration
private static void AppendRow(StringBuilder builder, CsvDumper.CsvCommand row)
Parameters
Type Name Description
StringBuilder builder
CsvDumper.CsvCommand row

ConfigureInput(IServiceProvider, string, string)

Factory helper for dependency-injection scenarios.

Declaration
public static CsvDumper ConfigureInput(IServiceProvider sp, string filePath, string fileName)
Parameters
Type Name Description
IServiceProvider sp

Service provider used to resolve dependencies.

string filePath

Directory path where the CSV file will be created.

string fileName

Base file name (without extension).

Returns
Type Description
CsvDumper

A new CsvDumper instance.

DisposeAsync()

Completes the queue and waits for accepted rows and barriers to finish. Repeatable.

Declaration
public ValueTask DisposeAsync()
Returns
Type Description
ValueTask

Enqueue(double, Vector<double>)

Enqueues a CSV row consisting of a timestamp and a MathNet.Numerics.LinearAlgebra.Vector<T>.

Declaration
public void Enqueue(double timestamp, Vector<double> vector)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

Vector<double> vector

Vector whose elements are written as subsequent columns.

Enqueue(double, IEnumerable<double>?)

Enqueues a CSV row consisting of a timestamp and an enumerable of values.

Declaration
public void Enqueue(double timestamp, IEnumerable<double>? values)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

IEnumerable<double> values

Values written as subsequent columns. If null, only the timestamp column is written.

Enqueue(double, double)

Enqueues a CSV row consisting of a timestamp and a single value.

Declaration
public void Enqueue(double timestamp, double value)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

double value

Sample value written as the second column.

Enqueue(double, params double[])

Enqueues a CSV row consisting of a timestamp and a set of values.

Declaration
public void Enqueue(double timestamp, params double[] values)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

double[] values

Values written as subsequent columns.

Enqueue(double, object)

Enqueues a CSV row by inspecting the runtime type of value and formatting it appropriately.

Declaration
public void Enqueue(double timestamp, object value)
Parameters
Type Name Description
double timestamp

UNIX-like timestamp (seconds) written as the first column.

object value

The published value. Supported types: MathNet.Numerics.LinearAlgebra.Vector<T>, double[], double, (string, Vector<double>), (string, double[]), MathNet.Numerics.LinearAlgebra.Matrix<T> (one row per matrix row), and IEnumerable<T> of double. Unsupported types are silently skipped.

Remarks

This is the primary entry point used by Publish(object) for automatic CSV logging. It dispatches to the appropriate typed Enqueue overload based on runtime type checks, ordered from most-specific to least-specific to avoid ambiguity.

For labelled tuples (e.g., from OnlineLDA), the label string is written as the second column (CSV-escaped), followed by the numeric values.

MathNet.Numerics.LinearAlgebra.Matrix<T> values produce multiple CSV rows (one per matrix row), all sharing the same timestamp. This is intended for future batch-output blocks.

Enqueue(double, ReadOnlySpan<double>)

Enqueues a CSV row consisting of a timestamp and a span of values.

Declaration
public void Enqueue(double timestamp, ReadOnlySpan<double> values)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

ReadOnlySpan<double> values

Values written as subsequent columns.

EnqueueCommand(CsvCommand)

Declaration
private void EnqueueCommand(CsvDumper.CsvCommand command)
Parameters
Type Name Description
CsvDumper.CsvCommand command

EnqueueLabelled(double, string, Vector<double>)

Enqueues a labelled CSV row: timestamp, label, v₀, v₁, …

Declaration
public void EnqueueLabelled(double timestamp, string label, Vector<double> vector)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

string label

String label written as the second column (CSV-escaped).

Vector<double> vector

Vector whose elements are written as subsequent columns.

EnqueueLabelled(double, string, double[])

Enqueues a labelled CSV row: timestamp, label, v₀, v₁, …

Declaration
public void EnqueueLabelled(double timestamp, string label, double[] values)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column.

string label

String label written as the second column (CSV-escaped).

double[] values

Array whose elements are written as subsequent columns.

EnqueueMatrix(double, Matrix<double>)

Enqueues one CSV row per matrix row, all sharing the same timestamp.

Declaration
public void EnqueueMatrix(double timestamp, Matrix<double> matrix)
Parameters
Type Name Description
double timestamp

Timestamp value written as the first column of each row.

Matrix<double> matrix

Matrix whose rows are each written as a separate CSV line: timestamp, row, col₀, col₁, ….

Remarks

The second column is the zero-based row index to allow reconstruction of the matrix shape.

EscapeCsv(string)

Escapes a string for inclusion in a CSV cell.

Declaration
private static string EscapeCsv(string s)
Parameters
Type Name Description
string s

Input string.

Returns
Type Description
string

The CSV-escaped string, quoted if needed and with embedded quotes doubled.

FlushAsync(CancellationToken)

Writes a non-droppable barrier after the rows accepted so far and awaits their flush. Cancellation abandons the wait, not rows already accepted. Writer failures propagate.

Declaration
public ValueTask FlushAsync(CancellationToken cancellationToken = default)
Parameters
Type Name Description
CancellationToken cancellationToken
Returns
Type Description
ValueTask

RunWriterAsync()

Background writer loop that drains the internal queue and appends lines to the file.

Declaration
private Task RunWriterAsync()
Returns
Type Description
Task

WritePendingAsync(StreamWriter, StringBuilder)

Declaration
private static Task WritePendingAsync(StreamWriter writer, StringBuilder batch)
Parameters
Type Name Description
StreamWriter writer
StringBuilder batch
Returns
Type Description
Task

WriterLoop()

Declaration
private Task WriterLoop()
Returns
Type Description
Task

Implements

IAsyncDisposable

Search documentation

Start typing to search the documentation.