feat(import): Add Splitwise CSV Import (Stacked on #472) - #483
Open
Uli-Z wants to merge 31 commits into
Open
Conversation
- Introduce ImportFormat interface and in-memory registry for adapters - Add registry helper to detect formats and delegate parsing - Add file import builder to parse, collect errors, and compute participant summaries via balances - Establish clear types for parsed group meta (name, currency, participants)
- Implement robust detection on full JSON payload with minimal structure checks - Parse export into ExpenseFormValues; coerce amounts/dates and validate against schema - Aggregate per-row errors and expose optional group meta (name, currency, participants) - Self-register adapter in the global registry
- Add marker-based debug format (DEBUG_IMPORT/DEBUG_ERRORS) with unambiguous detection - Emit one error per line for quick UI testing of failure paths - Include simple fixture file for manual verification - Register debug adapter with registry at low priority
…finalize) - Expose preview endpoint to parse and summarize uploaded file before import - Implement job-based create flow with chunked processing and progress reporting - Provide cancel/cleanup and finalize endpoints to control lifecycle - Register endpoints under groups router
- Dropzone with drag-and-drop and accessible labeling - Analysis panel to display detected format, totals and errors - Progress view for chunked import with visual bar - Result view to confirm completion or cancellation
- Combine upload + preview + scroll-to-confirm + chunked import in one dialog - Handle cancel/finalize flows with toasts and resilient state reset - Support optional prefill of group name from parsed file meta - Integrate TRPC mutations with defensive error handling
- Add Import from file option to create menu - Mount FileImportModal and navigate to new group on success - Persist created group to recent list and refresh view
- Add strings for upload, preview errors, progress, and results - Provide German, English, Spanish and French localizations - Wire keys used across import components and modal
…ility Extracted complex parsing logic from 'parseToInternal' into smaller, private helper methods for better readability and easier maintenance.
Implemented a daily cleanup of ImportJob records older than 24 hours at the start of a new import, preventing database bloat from stale jobs.
Implemented Zod validation for 'expensesToCreate' in the ImportJob model when processing chunks. This ensures data integrity and prevents runtime errors from corrupted job data by safely parsing and validating the JSON.
…factoring This commit consolidates the review feedback implementation: Security & Robustness: - Enforced a 10MB limit on file uploads to prevent DoS. - Implemented optimistic locking in chunk processing to prevent race conditions. Refactoring: - Extracted UI logic into 'useFileImportProcess' hook for better separation of concerns. - Centralized participant derivation logic in the import library. Quality: - Added integration tests for category mapping consistency. - Applied code formatting.
- Updated ImportFormat interface to be async (Promise-based) to support non-blocking operations and future worker offloading. - Moved file preview logic from server-side tRPC to client-side. - Removed fs dependencies from import parsers to allow browser execution. - Removed obsolete importFromFilePreview tRPC endpoint. - Fixed type errors and updated tests to align with async interface.
Adds a new tRPC procedure 'processBatch' that accepts a list of expenses and inserts them transactionally. This enables client-side batching strategies.
Moves the import orchestration to the client to reduce server load and complexity. - Implements batching logic (processing 50 expenses at a time). - Adds a 10MB file size limit validation. - Includes logic to correctly map participant names to UUIDs during the import process, ensuring expenses are linked to the correct users even if the source file uses names.
…cedures Removes the stateful 'ImportJob' model and associated procedures (start-job, run-chunk, etc.). Previously, the server tracked import progress in the database, which caused excessive I/O overhead. The process is now stateless on the server, relying on the new client-driven batching approach.
9 tasks
|
Thanks @Uli-Z for your contribution, sad to see this not get through. I have created and main a fork of the original work on Spliit living at https://github.com/antonio-ivanovski/spliit-cloud and hosted https://spliit.cloud/ Going through your PR made me aware that the Splitwise export is localized in the current Splitwise language. Will be making this change to my imported now. Thanks for the hint. Meanwhile, give spliit.cloud a try and let me know how you find the forked version. |
5 tasks
Ecklebe
pushed a commit
to Ecklebe/spliit
that referenced
this pull request
Sep 4, 2026
Upstream spliit-app#483 is stacked on spliit-app#472, the file-import infrastructure this fork already merged, so 34 of its 46 files are things we have. Rather than merge the PR - which is two months stale and no longer mergeable upstream - this lifts only the self-contained adapter under src/lib/imports/formats/splitwise/ and registers it with one side-effect import, the same way spliit-json is registered. Upstream cannot take spliit-app#483 until spliit-app#472 lands; this fork can. Two adaptations were needed, both because spliit-app#472 has moved on here since spliit-app#483 was written against it: - ExpenseFormValues now requires `location` (fork PR spliit-app#172). A Splitwise export has no location, so both emit sites pass null. - More importantly, the adapter emitted its internal index-based participant ids (`p0`, `p1`) as paidBy/paidFor. Our import procedure remaps by *display name* - "adapters set participants to display names", which is also what the Spliit-JSON adapter emits - so importing a real file failed outright with "Invalid participant ID: p0". The reconstruction still works in indices, so that two participants sharing a name stay distinct through the delta maths; it now converts back to names at the parseToInternal boundary. Its tests were asserting the internal ids, and now assert names. Also adds src/lib/imports/registry.test.ts. Each adapter had its own tests, but nothing covered the registry picking between them, which is the part that breaks when a format is added: Splitwise CSV and Spliit JSON must reject each other's files outright rather than merely score lower, and an unrelated file must select nothing at all. Verified end to end against the development cluster, not just in unit tests - a three-row export imported and reconstructed correctly, including the case worth checking: Splitwise row Spliit result 30.00, Alice +20 / Bob -10 / C -10 Alice pays 3000, split 1000 each 30.00, Alice +5 / Bob +5 / C -10 two 1500 expenses (Spliit cannot express two payers); nets to +5/+5/-10 Payment 25.00, Bob +25 / C -25 reimbursement, Bob -> Charlie 2500 The all-or-nothing behaviour of groups.importFromFile also held: the shipped fixture contains deliberately invalid rows and the whole file is rejected, which is the documented policy for that procedure. 345 tests across 26 suites, tsc and prettier clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PYTybS1LgmN9uxQxsx7EQe
Ecklebe
pushed a commit
to Ecklebe/spliit
that referenced
this pull request
Sep 4, 2026
… upstream PRs Three strands, all verified against the development cluster rather than only in tests. Guarding against upstream spliit-app#618. Prisma 7's pg driver adapter no longer defaults to the `postgres` database, so a connection URL with no database path fails with "The table public.Group does not exist". Both .env.example and the cluster compose now name it explicitly. Shrinking the merge surface, after v1.23.1 cost 50 conflicting files. The schema becomes a folder (upstream's models stay in a file git still scores as a rename), fork translations move to messages/fork/ layered by the deepmerge i18n already used, the fork's env fields move to env.fork.ts, and the login/sync/admin seams in upstream files become single calls. The measured effect on files upstream also owns: 55 -> 53 files and +2406/-581 -> +1663/-491, a 31% cut in the lines that can conflict, with all six translation catalogues now byte-identical to upstream. Adopting three PRs that fit this fork. Splitwise CSV import (spliit-app#483, adapter only - it is stacked on the spliit-app#472 we already carry, so upstream cannot take it yet), document links in the JSON export as exportVersion 3 (spliit-app#554, exporting our own /api/documents path rather than the private bucket URL), and the monthly category spending visuals (spliit-app#555) that replace the spliit-app#532 the v1.23.1 merge had to drop. 352 unit tests across 27 suites, and the Playwright suite is 54 passed / 0 failed at one worker - green for the first time here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PYTybS1LgmN9uxQxsx7EQe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This PR implements the Splitwise Import (closes #22), building on the generic import infrastructure from #472. It enables users to migrate their group history from Splitwise to Spliit.
The Challenge
As discussed in #22, Splitwise exports provide a ledger of balance changes ("User A paid 50, User B owes 50") rather than the original split configuration. This means the original intent (how exactly an expense was split) is often lost.
Implementation
1. Reconstruction Algorithm
The core of this PR is a deterministic algorithm that reconstructs plausible expenses from the balance deltas.
2. Localization & Future Scope
The adapter currently supports imports in English and German.
Verification
Automated tests (
index.test.ts) verify that the reconstruction logic holds up against complex scenarios (multi-payer, uneven splits, self-payments) and preserves the mathematical integrity of the group balances.