Introduction

This collection of projects demonstrates that DISQLite3 is well capable to

handle 20 million in very reasonable time. DISQLite3 by far outperforms other
Delphi database systems and even rivals big database players like MySql.

The individual projects in this folder show how to generate data, export to CSV,
and import from CSV. All projects handle the full number of 20 million records.

Requirements

* DISQLite3, DISQLite3 Pro required for Mem_Helper projects.

* DIUnicode for CSV import.

How to Run the Tests

1. Compile and run one of the Generate projects. They will create a
20_million_generate.db3 database file filled with random data.

2. Compile and run the Export project. This will export all 20 million records
from the generated table to a newly created CSV file.

3. Compile and run one of the Import projects. They will import all 20 million
records from the CSV file to a newly generated 20_million_import.db3 database.

Further Performance Improvements

The projects are set up to use no more than about 450 MB of memory for database
operations. Using more memory will speed up insertions and index creation. Refer
to the source code comments for details.

About 20% improvement can be achieved by using a fast memory manager like FastMM.

Approximate Timings

Generate with memory helper table: 11 Minutes

CSV export: 8 Minutes
Import with memory helper table: 13 Minutes

All timings measured on a Intel Core Duo 1.8 MHz, 1 GB memory, 5200 rpm hard
disk notebook system. Systems with faster hard disks and server hardware are
expected to show even better performance.