Clarion

Clarion is a 4GL programming language and database system known for rapid application development. It has been used extensively for business applications, particularly in vertical market software.

What You Can Upload

• .TPS TopSpeed database files
• .DAT Clarion data files, with their .MEM memo files
• ZIP archive containing complete file sets

What You Get Out

DataMeans converts your data into multiple modern formats:

How to Export / Obtain Files

1. Locate your Clarion application's data directory
2. Copy the .TPS or .DAT data files
3. Include the .MEM memo file for each .DAT that has one, with the same name as its .DAT (CUSTOMER.DAT and CUSTOMER.MEM)
4. Create a ZIP archive with all files
5. Upload the ZIP to DataMeans

Supported Features

• TopSpeed .TPS and Clarion .DAT files
• TopSpeed files checked value for value against two independent TopSpeed readers on real files, and Clarion 2.x .DAT files checked against a real sample set that includes owned and encrypted files
• Field names and types from the files themselves, with no dictionary needed
• A TopSpeed table the file does not name (every single-table file: it is called "UNNAMED") takes the name of the file you upload
• A preview that shows the real columns, the exact number of records and the first rows of each table (for TopSpeed files up to 64 MB)
• Rows in the order your application added them
• Negative packed DECIMAL amounts keep their sign, in TopSpeed and .DAT files alike, and TopSpeed 32-bit floating-point values are shown as written (0.1, not 0.10000000149011612)
• Memo fields as text, in a column of their own after the record's other fields: TopSpeed memos from the file itself, .DAT memos from the .MEM file with the same name (each .DAT in a ZIP gets its own). TopSpeed BLOB fields as base64 bytes
• .DAT files marked as owned or encrypted (Clarion's OWNER and ENCRYPT attributes) are read without a password: the format keeps the key that scrambles them in the file itself
• A .DAT table is named after its file, where the file only holds the label of its record (RECORD)
• TopSpeed arrays (DIM) become one column per element (CODE_1, CODE_2, ...). A GROUP becomes its fields, and a group that is itself an array has its fields repeated for each instance (SKU_1, QTY_1, SKU_2, QTY_2)
• Clarion dates and times converted to standard dates and times

Known Limitations

• Keys and indexes (.K* files) are not carried over
• Password-protected (encrypted) TopSpeed .tps files cannot be read: remove the password in Clarion first. A password-protected .tps has no readable header, so DataMeans reports that this is what it most likely is (or that the file is damaged) rather than that it is not a Clarion file
• A .DAT with a DECIMAL of an even number of digits (for example DECIMAL(10,2)) is read with its spare half-byte as a leading zero; the sample files only have odd digit counts, so check such a column against your application
• In TopSpeed files, a time of exactly zero is read as blank: the file stores midnight and "no time" the same way
• In TopSpeed files, a date or time that is not a real one (month 13, 30 February) is kept exactly as stored, in a text column, and the report names it
• NaN and infinite floating-point values in TopSpeed files cannot be carried to the outputs: they are left empty and the report says how many
• Some legacy character encodings may need adjustment
• Very large tables (1M+ rows) may take extended processing time

Note on the Excel formats. An .xlsx sheet holds at most 1,048,575 rows of data and 16,384 columns, and a legacy .xls sheet 65,535 rows and 256 columns. Rather than ship a file that looks complete but is silently cut short, DataMeans leaves a table out of any Excel format it does not fit and notes it in the job report. The .xlsx copy is also left out for tables of more than 8,000,000 cells (1,048,575 rows of 12 columns is 12.6 million), which take too long and too much memory to write, and the .xls copy for tables of more than 150,000 cells, which the legacy format takes far too long to write. The CSV, JSON and SQL outputs always contain every row.

Overview

Clarion is a fourth-generation programming language and database development environment designed for rapid application development. It includes proprietary ISAM file formats and a file-driver layer that also connects applications to external file formats and SQL databases. Clarion emphasizes visual development with template-driven code generation and centralized data dictionary management.

History and Background

• 1986: Clarion 1.0 released for DOS by Bruce Barrington's Barrington Systems, using the proprietary .DAT ISAM format.
• 1988: Clarion Professional Developer (CPD) 2.0 introduces the Designer, a data dictionary, and model-file code generation.
• 1992: Clarion Software merges with compiler maker Jensen & Partners International (JPI); the combined company becomes TopSpeed Corporation.
• 1993: Clarion Database Developer (CDD) 3.0 replaces model files with extensible templates and compiles to native machine code.
• 1995: Clarion for Windows 1.0 ships in January; the single-file TopSpeed (.TPS) format debuts the same year with Clarion for DOS 3.1.
• 1997: Clarion 4 introduces the Application Builder Classes (ABC) template set and class library.
• 1998: Clarion 5 released, splitting the product into Professional and Enterprise editions.
• 2000: The Clarion product line moves from TopSpeed Corporation to SoftVelocity; Clarion 5.5 adds interfaces to the language's OOP support.
• 2003: Clarion 6.0 adds preemptive multithreading.
• 2009: Clarion 7 ships with a new SharpDevelop-based IDE and an MSBuild-based build system.
• 2011: Clarion 8 released in September, continuing the Clarion 7 series.
• 2014: Clarion 9.1 reaches gold release in August.
• 2015: Clarion 10 released, with the IDE reworked to run on .NET Framework 4.
• 2018: Clarion 11 released with faster builds and touch-input support.
• 2025: Clarion 12 released.

File Format Specifications

Database Files:

• .TPS: TopSpeed database files (primary format; data, keys, and memos in one file)
• .DAT: Clarion DOS data files (original ISAM format)
• .Knn: Index files accompanying .DAT databases (.K01, .K02, etc.)
• .MEM: Memo field storage accompanying .DAT databases

File Structure:

• Single-file design: data records, memo data, key indexes, and table definitions stored together
• File header occupying the first 0x200 bytes, with the tOpS signature at offset 0x0E
• Page-based storage, with pages aligned on 256-byte boundaries and grouped into blocks
• Optional run-length encoding (RLE) compression of pages, applied when a page's stored length differs from its uncompressed length
• Records tagged by type code: 0xF3 for data records, 0xFA for table definitions, 0xFC for memos
• Table-definition records describing each field's offset, length, and data type
• Multi-byte values stored little-endian, except record numbers, which are big-endian
• Optional encryption of file contents

Key Components:

• Data dictionary: Centralized schema management
• Templates: Code generation for common patterns
• Applications: Generated programs compiled to native executables
• Reports: Formatted output definitions
• Forms: User interface designs

Data Types and Structures

Database Objects:

• Tables: Primary data containers with fields and records
• Keys: Indexed fields for fast record access
• Relationships: Parent-child links with referential integrity rules defined in the dictionary
• Queues: In-memory data structures
• Views: Virtual composites of related files supporting join operations

Version Differences

Compatibility Notes:

• Clarion 7 and later are backward compatible with releases back to Clarion for Windows 1.5
• The TopSpeed driver stores data, memos, and all key indexes in a single disk file
• Applications predating the TopSpeed driver use the .DAT/.Knn/.MEM file set
• File drivers also access Btrieve, dBase, FoxPro, and Paradox files, plus SQL databases via ODBC and native drivers
• 16-bit executable targets were dropped with Clarion 6.0; Clarion 6.3 was the final 16-bit IDE
• The four-byte DATE and TIME field types match the corresponding Btrieve Record Manager field formats
• Clarion standard dates count days elapsed since December 28, 1800 (valid range January 1, 1801 to December 31, 9999); standard times count hundredths of a second since midnight, plus one

Technical References

• SoftVelocity Clarion
• Wikipedia: Clarion (programming language)
• Liberating TPS Files: TopSpeed Format Analysis
• A History of the Clarion Language (Sterling Data)
• ClarionHub Community