nddipiazza opened a new pull request, #2987: URL: https://github.com/apache/tika/pull/2987
## Summary Fixes [TIKA-4798](https://issues.apache.org/jira/browse/TIKA-4798): `dcterms:modified` for `.msg` (Outlook MAPI) files changed between Tika 2.9.0 and 3.2.3. ## Root Cause [TIKA-4360](https://issues.apache.org/jira/browse/TIKA-4360) (#2073, merged into the 3.x line in Dec 2024) added code to `OutlookExtractor.handleGeneralDates()` that unconditionally overwrites `TikaCoreProperties.MODIFIED` (`dcterms:modified`) with the raw MAPI `PR_LAST_MODIFICATION_TIME` property whenever it is present. `PR_LAST_MODIFICATION_TIME` records when the `.msg` file's underlying CFB/OLE storage was last **written to disk** (e.g. when Outlook re-saves or exports the item as a `.msg` file) — it is not a semantic "the email content was edited" timestamp. Since an email's content is effectively immutable once sent, this property commonly reflects a later, unrelated storage-level event (archival, export, re-indexing, etc.), not a real content modification. I confirmed this against the reporter's attached sample files: `PR_LAST_MODIFICATION_TIME` for `Sample1.msg` is `2018-10-06T00:03:21Z`, exactly matching the "wrong" 3.2.3 value reported in the ticket, while `msg.getMessageDate()` (delivery/submit time, used in 2.9.0) is `2018-08-03T19:13:22Z`, matching the "expected" 2.9.0 value. ## Changes - Removed the block in `OutlookExtractor.handleGeneralDates()` that overwrote `dcterms:modified` with `PR_LAST_MODIFICATION_TIME`. `dcterms:modified` now falls back to the message date (delivery/submit time) again, matching 2.9.0 behavior. - The raw property is **not lost** — it remains available unchanged under `mapi:last-modification-time` (already populated separately by `handleMessageInfo`/`LITERAL_TIME_PROPERTIES`, added in the same TIKA-4360 change). - Added a regression assertion in `OutlookParserTest#testOutlookParsing` using the existing `test-outlook.msg` resource, whose `PR_LAST_MODIFICATION_TIME` diverges from its message date by about six months — the same kind of divergence reported in the ticket. ## Review Focus Areas - Confirm the semantics: should `dcterms:modified` reflect message date or `PR_LAST_MODIFICATION_TIME`? This PR takes the position that message date is the more meaningful default for email, given `PR_LAST_MODIFICATION_TIME`'s storage-level semantics, while still exposing the raw MAPI value separately. - Check no other code path or downstream consumer relies on the TIKA-4360 override behavior. ## Critical Files - `tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/OutlookExtractor.java` - `tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module/src/test/java/org/apache/tika/parser/microsoft/OutlookParserTest.java` ## Testing Instructions ``` cd tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-microsoft-module mvn test -Dtest=OutlookParserTest ``` Also ran the full module test suite (`mvn test`) with no new failures, and manually parsed the reporter's attached sample `.msg` files to confirm `dcterms:modified` now matches the value produced by Tika 2.9.0. ## Review Checklist - [x] Root cause identified and confirmed against reporter's sample files - [x] Existing tests pass - [x] New regression test added - [x] No data loss — raw MAPI property still exposed under its own key ## Potential Concerns - This is a behavior change from 3.2.3 back to 2.9.0 semantics; any user who started relying on the 3.x `dcterms:modified` value (intentionally or not) will see it change again. Given this was an undocumented, likely unintended side effect of TIKA-4360, I believe reverting to the original, more intuitive semantics is the right call, with the raw property still available under `mapi:last-modification-time` for anyone who wants it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
