Repository navigation
Read the text of Access 97 files in the code page of the database - #101
Merged
Merged
Conversation
- Add japaneseTestV1997.mdb, an Access 97 database in code page 932 made with DAO 3.6, with Japanese names of a table, its columns, an index and a query, a Japanese column description, and Japanese values, among them half-width katakana and characters that some converters map differently - Add the script that made it, jet3-japanese-generator.vbs, and describe in SOURCES.md how it was made Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Access 97 (Jet3) stores text in the code page of the database, but it was always read as Latin-1: Japanese (code page 932) came out garbled, and in Western files (1252) characters such as the euro sign and the trademark sign came out as controls - Add TextEncoding, how a file stores text, which PageReader::text_encoding tells from the code page in the header, and read the values, memos, column and index names, properties and Access 97 macros of a Jet3 file in that code page, or as Latin-1 when jetdb does not know it - Take TextEncoding in place of is_jet3 where text is read, which changes the arguments of entry_properties and encoding::decode_text, and replaces TableDef::is_jet3 with TableDef::text_encoding - Add tests that the Japanese names, values, description and query of japaneseTestV1997.mdb, and the code page 1252 characters of testIndexCodesV1997.mdb, read as they should Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… the database - Say in README.md and README.ja.md that Jet3 text is in the code page of the database, not Latin-1 - Say after the table of supported versions in docs/lib.md, docs/lib.ja.md and the crate documentation that jetdb reads Jet3 text in the code page in the header, and as Latin-1 when it does not know the code page Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Move the reading of a single-byte string, in the code page or as Latin-1, into single_byte_string, which takes the closure that reads the strings of a row from a cyclomatic complexity of 10 to 7 - Correct the name of the argument the documentation of parse_macro_grid still gave Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PageReader::text_encodingtells from the header how a file stores text, as the newTextEncoding: in the code page of the header for Jet3, in UTF-16LE later. The values, memos, column and index names, properties and Access 97 macros of a Jet3 file are read in that code page withencoding_rs, already a dependency, and as Latin-1 when jetdb does not know the code page. mdbtools and Jackcess read the code page of the header too. In the Jet3 test files, the code page of every text and memo column agrees with the header.TextEncodingtakes the place ofis_jet3where text is read. This changes the arguments ofentry_propertiesandencoding::decode_text, and replacesTableDef::is_jet3withTableDef::text_encoding, breaking changes for 0.4.0.testdata/V1997/japaneseTestV1997.mdbis a new test database in code page 932, made with DAO 3.6 bytestdata/jet3-generators/jet3-japanese-generator.vbs, as Access 2013 and later cannot create Access 97 files. New tests check that its Japanese table, column, index and query names, column description and values read as they were written, half-width katakana and①～髙included, and that the code page 1252 characters oftestIndexCodesV1997.mdbread as such. Reading as Latin-1 again makes both fail.README.md/README.ja.md,docs/lib.md/docs/lib.ja.mdand the crate documentation describe how Jet3 text is read.🤖 Generated with Claude Code