| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a new search qualifier `is:broken` that allows users to filter bookmarks
with broken or failed links. This matches the functionality on the broken links
settings page, where a link is considered broken if:
- crawlStatus is "failure"
- crawlStatusCode is less than 200
- crawlStatusCode is greater than 299
The qualifier supports negation with `-is:broken` to find working links.
Changes:
- Add brokenLinks matcher type definition
- Update search query parser to handle is:broken qualifier
- Implement query execution logic for broken links filtering
- Add autocomplete support with translations
- Add parser tests
- Update search query language documentation
Co-authored-by: Claude <noreply@anthropic.com>
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* feat: Add automated bookmark backup system
Implements a comprehensive automated backup feature for user bookmarks with the following capabilities:
Database Schema:
- Add backupSettings table to store user backup preferences (enabled, frequency, retention)
- Add backups table to track backup records with status and metadata
- Add BACKUP asset type for storing compressed backup files
- Add migration 0066_add_backup_tables.sql
Background Workers:
- Implement BackupSchedulingWorker cron job (runs daily at midnight UTC)
- Create BackupWorker to process individual backup jobs
- Deterministic scheduling spreads backup jobs across 24 hours based on user ID hash
- Support for daily and weekly backup frequencies
- Automated retention cleanup to delete old backups based on user settings
Export & Compression:
- Reuse existing export functionality for bookmark data
- Compress exports using Node.js built-in zlib (gzip level 9)
- Store compressed backups as assets with proper metadata
- Track backup size and bookmark count for statistics
tRPC API:
- backups.getSettings - Retrieve user backup configuration
- backups.updateSettings - Update backup preferences
- backups.list - List all user backups with metadata
- backups.get - Get specific backup details
- backups.delete - Delete a backup
- backups.download - Download backup file (base64 encoded)
- backups.triggerBackup - Manually trigger backup creation
UI Components:
- BackupSettings component with configuration form
- Enable/disable automatic backups toggle
- Frequency selection (daily/weekly)
- Retention period configuration (1-365 days)
- Backup list table with download and delete actions
- Manual backup trigger button
- Display backup stats (size, bookmark count, status)
- Added backups page to settings navigation
Technical Details:
- Uses Restate queue system for distributed job processing
- Implements idempotency keys to prevent duplicate backups
- Background worker concurrency: 2 jobs at a time
- 10-minute timeout for large backup exports
- Proper error handling and logging throughout
- Type-safe implementation with Zod schemas
* refactor: simplify backup settings and asset handling
- Move backup settings from separate table to user table columns
- Update BackupSettings model to use static methods with users table
- Remove download mutation in favor of direct asset links
- Implement proper quota checks using QuotaService.checkStorageQuota
- Update UI to use new property names and direct asset downloads
- Update shared types to match new schema
Key changes:
- backupSettingsTable removed, settings now in users table
- Backup downloads use direct /api/assets/{id} links
- Quota properly validated before creating backup assets
- Cleaner separation of concerns in tRPC models
* migration
* use zip instead of gzip
* fix drizzle
* fix settings
* streaming json
* remove more dead code
* add e2e tests
* return backup
* poll for backups
* more fixes
* more fixes
* fix test
* fix UI
* fix delete asset
* fix ui
* redirect for backup download
* cleanups
* fix idempotency
* fix tests
* add ratelimit
* add error handling for background backups
* i18n
* model changes
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
| |
|
|
|
| |
* fix: Propagate group ids
* fix tests
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* refactor(trpc): extract rate limiter into dedicated plugin
Move the rate limiting middleware from the trpc package to the
centralized plugins package. This improves code organization by
consolidating all plugins in a single location.
Changes:
- Created packages/plugins/trpc-ratelimit/ plugin
- Moved rate limiter from packages/trpc/rateLimit.ts to
packages/plugins/trpc-ratelimit/src/index.ts
- Added trpc-ratelimit export to plugins package.json
- Added @trpc/server dependency to plugins package
- Updated trpc package to import from @karakeep/plugins/trpc-ratelimit
- Added @karakeep/plugins dependency to trpc package
- Removed packages/trpc/plugins/ directory
* refactor(plugins): decouple rate limiter from tRPC
Refactor the rate limiting plugin to be framework-agnostic, allowing
it to be used outside of tRPC contexts. The plugin now has a generic
core with a tRPC-specific adapter.
Changes:
- Renamed trpc-ratelimit to ratelimit plugin
- Created generic RateLimiter class with framework-agnostic API
- Added checkRateLimit() method that returns allow/deny results
- Created separate tRPC adapter (src/trpc.ts) that uses the generic core
- Exported both generic (RateLimiter, globalRateLimiter) and
tRPC-specific (createRateLimitMiddleware) APIs
- Updated trpc package to import from @karakeep/plugins/ratelimit
- Updated plugins package.json exports
Benefits:
- Rate limiter can now be used in any context (HTTP handlers, WebSocket, etc.)
- Cleaner separation of concerns
- Easy to create adapters for other frameworks
- Generic API allows for custom error handling
* refactor(plugins): integrate rate limiter with plugin registry
Refactor the rate limiting plugin to use the centralized plugin
system with PluginManager, making it consistent with other plugins
like queue and search providers.
Changes:
- Added RateLimit plugin type to PluginType enum
- Created RateLimitClient interface in packages/shared/ratelimiting.ts
- Created RateLimitProvider class implementing PluginProvider
- Updated plugin to auto-register with PluginManager on import
- Updated tRPC adapter to use getRateLimitClient() from PluginManager
- Added ratelimit plugin to loadAllPlugins() in shared-server
- Updated shared/plugins.ts with RateLimit type mapping
Benefits:
- Consistent plugin architecture across the codebase
- Rate limiter can be swapped with alternative implementations
- Centralized plugin management and logging
- Better separation of concerns
- Framework-agnostic core with tRPC adapter pattern
* refactor(trpc): move rate limit middleware to trpc package
Move the tRPC-specific rate limiting middleware from the plugins
package to the trpc package, making the plugins package
framework-agnostic.
Changes:
- Moved packages/plugins/ratelimit/src/trpc.ts to
packages/trpc/lib/rateLimit.ts
- Updated packages/trpc/index.ts to import from local lib/rateLimit
- Removed tRPC export from packages/plugins/ratelimit/index.ts
- Removed @trpc/server dependency from packages/plugins/package.json
Benefits:
- plugins package is now framework-agnostic
- tRPC-specific code lives in the trpc package where it belongs
- Cleaner separation of concerns
- Rate limiter plugin can be used in any context without tRPC
* refactor(plugins): rename to ratelimit-memory and add tests
Rename the rate limiting plugin from "ratelimit" to "ratelimit-memory"
to better indicate it's an in-memory implementation. This naming leaves
room for future implementations like ratelimit-redis. Also added
comprehensive test coverage.
Changes:
- Renamed packages/plugins/ratelimit to ratelimit-memory
- Updated package.json export from ./ratelimit to ./ratelimit-memory
- Updated shared-server to import @karakeep/plugins/ratelimit-memory
- Added comprehensive unit tests (index.test.ts):
- Rate limit enforcement tests
- Window expiration tests
- Identifier and path isolation tests
- Reset functionality tests
- Cleanup mechanism tests
- Added provider integration tests (provider.test.ts):
- PluginProvider interface compliance
- Client singleton behavior
- End-to-end rate limiting functionality
Benefits:
- More descriptive plugin name indicating the storage mechanism
- Better test coverage ensuring reliability
- Easier to add alternative implementations (Redis, etc.)
* change the api to only take the key
* move the serverConfig check to the trpc
* fix lockfile
* get rid of the timer
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* feat: add user file upload support for bookmarks
Add a new "user-uploaded" asset type that allows users to upload and
attach their own files to bookmarks from the attachment box in the
bookmark preview page.
Changes:
- Add USER_UPLOADED asset type to database schema
- Add userUploaded to zAssetTypesSchema for type safety
- Update attachment permissions to allow attaching/detaching user files
- Add fileName field to asset schema for displaying custom filenames
- Add "Upload File" button in AttachmentBox component
- Display actual filename for user-uploaded files
- Allow any file type for user uploads (respects existing upload limits)
- Add Upload icon for user-uploaded files
Fixes #1863 related asset attachment improvements
* fix: ensure fileName is returned and remove edit button for user uploads
- Fix attachAsset mutation to fetch and return complete asset with fileName
instead of just returning the input (which lacks fileName)
- Remove replace/edit button for user-uploaded files - users can only
delete and re-upload instead
- This ensures the filename displays correctly in the UI immediately
after upload
Fixes fileName propagation issue for user-uploaded assets
* fix asset file name
* remove filename from attach asset api
---------
Co-authored-by: Claude <noreply@anthropic.com>
|
| | |
|
| |
|
|
|
|
|
| |
* fix(search): include link titles in title matcher
* docs(search): add title qualifier
* docs: remove title qualifier from v0.27 guide
|
| | |
|
| |
|
|
|
|
|
|
| |
This commit allows the following mime types to be uploaded and attached
as video assets on bookmarks.
- video/mp4
- video/webm
- video/x-matroska
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
* refactor: Move bookmark utils from shared-react to shared
* Expose RSS feeds for lists
* Add e2e tests
* Slightly improve the look of the share dialog
* allow specifying a limit in the rss endpoint
|
| |
|
|
| |
This reverts commit 6178736d64180f9bc8954099c90d54aa2f9f35f5.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add schema for the new rule engine
* Add rule engine backend logic
* Implement the worker logic and event firing
* Implement the UI changesfor the rule engine
* Ensure that when a referenced list or tag are deleted, the corresponding event/action is
* Dont show smart lists in rule engine events
* Add privacy validations for attached tag and list ids
* Move the rules logic into a models
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Updated pdf2json to 3.1.5
* Extract and store a screenshot from PDF files using pdf2pic
* Installing graphicsmagick and ghostscript
* Generate Missing PDF screenshot with tidyAssets worker for backward support
* Display PDF screenshot instead of the PDF in web if it exists.
* Display PDF screenshot in mobile app if exists.
* Updated pnpm-lock.yaml
* Removed console.log
* Revert the unnecessary changes in package.json
* Revert pnpm-lock changes
* Prevent rendering PDF files if the screenshot is not generated
* refactor: replace useEffect with useMemo for section initialization
* feat: show PDF file download button and handle large PDFs by defaulting to screenshot view
* feat: add file size to openapi spec
* feature: Add Assets preprocessing in fix mode to admin actions
* i18n: add reprocess_assets_fix_mode translation
* i18n: Add missing ar translations
* A bunch of fixes
* Fix openspec schema
---------
Co-authored-by: Mohamed Bassem <me@mbassem.com>
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* feat: Add support for smart lists
* i18n
* Fix update list endpoint
* Add a test for smart lists
* Add header to the query explainer
* Hide remove from lists in the smart context list
* Add proper validation to list form
---------
Co-authored-by: Deepak Kapoor <41769111+orthdron@users.noreply.github.com>
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* feat: Implement search filtering in the backend
* feat: Implement search language parser
* rename matcher name
* Add ability to interleve text
* More fixes
* be more tolerable to parsing errors
* Add a search query explainer widget
* Handle date parsing gracefully
* Fix the lockfile
* Encode query search param
* Fix table body error
* Fix error when writing quotes
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Allow downloading more content from a webpage and index it #215
Added a worker that allows downloading videos depending on the environment variables
refactored the code a bit
added new video asset
updated documentation
* Some tweaks
* Drop the dependency on the yt-dlp wrapper
* Update openapi specs
* Dont log an error when the url is not supported
* Better handle supported websites that dont download anything
---------
Co-authored-by: Mohamed Bassem <me@mbassem.com>
|
| | |
|
| |
|
