File Registration

Ducklings supports loading data from various sources by registering virtual files.

Remote Files (URL)

Register a remote file by URL to access it in queries:

// Register a remote Parquet file
await db.registerFileURL(
  'remote.parquet',
  'https://example.com/data.parquet'
);

// Query the registered file
const rows = await conn.query("SELECT * FROM 'remote.parquet' LIMIT 10");
Copy

Parameters

registerFileURL(
  name: string,      // Virtual file name
  url: string,       // Remote URL
  protocol?: string, // 'HTTP' or 'HTTPS' (auto-detected)
  directIO?: boolean // Use direct I/O (default: false)
): Promise<void>
Copy

In-Memory Buffers

Register a Uint8Array as a virtual file:

// Create CSV data in memory
const csvData = new TextEncoder().encode('id,name\n1,Alice\n2,Bob');

// Register as virtual file
await db.registerFileBuffer('data.csv', csvData);

// Read with DuckDB's CSV reader
const rows = await conn.query("SELECT * FROM read_csv('data.csv')");
Copy

Use Cases

• Loading data fetched from custom sources
• Processing data from File API (browser file uploads)
• Working with generated data

// Example: Processing uploaded file
const fileInput = document.getElementById('file') as HTMLInputElement;
const file = fileInput.files[0];
const buffer = new Uint8Array(await file.arrayBuffer());

await db.registerFileBuffer(file.name, buffer);
const data = await conn.query(`SELECT * FROM read_csv('${file.name}')`);
Copy

Text Files

Register text content directly:

await db.registerFileText('config.json', JSON.stringify({ key: 'value' }));

const rows = await conn.query(`
  SELECT * FROM read_json('config.json')
`);
Copy

File Operations

Drop Files

// Remove a single file
await db.dropFile('data.csv');

// Remove all registered files
await db.dropFiles();
Copy

Export Files

// Export a file to a buffer
const buffer = await db.copyFileToBuffer('output.parquet');

// Copy a file to another path
await db.copyFileToPath('source.parquet', 'backup.parquet');
Copy

List Files

// List files matching a pattern
const files = await db.globFiles('*.parquet');
// [{ name: 'data.parquet', size: 1024 }, ...]
Copy

Flush Buffers

// Ensure all file buffers are written
await db.flushFiles();
Copy

Supported Formats

DuckDB can read many formats from registered files:

Format	Read Function	Example
CSV	read_csv()	SELECT * FROM read_csv('data.csv')
JSON	read_json()	SELECT * FROM read_json('data.json')
Parquet	Direct path	SELECT * FROM 'data.parquet'
NDJSON	read_ndjson_auto()	SELECT * FROM read_ndjson_auto('data.ndjson')

Complete Example

import { init, DuckDB } from '@ducklings/browser';

await init();
const db = new DuckDB();
const conn = await db.connect();

// Register remote Parquet file
await db.registerFileURL(
  'cities.parquet',
  'https://raw.githubusercontent.com/example/data/cities.parquet'
);

// Query remote data
const cities = await conn.query(`
  SELECT city, population
  FROM 'cities.parquet'
  WHERE country = 'US'
  ORDER BY population DESC
  LIMIT 10
`);

// Export results
await conn.execute(`
  COPY (SELECT * FROM 'cities.parquet' WHERE country = 'US')
  TO 'us_cities.parquet' (FORMAT PARQUET)
`);

// Get the exported file
const exportedData = await db.copyFileToBuffer('us_cities.parquet');

// Clean up
await db.dropFiles();
await conn.close();
await db.close();
Copy