aboutsummaryrefslogtreecommitdiffstats
path: root/apps/workers (follow)
Commit message (Collapse)AuthorAgeFilesLines
* feat(rss): Add import tags from RSS feed categories (#2031)Mohamed Bassem2025-11-021-0/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * feat(feeds): Add import tags from RSS feed categories - Add importTags boolean field to rssFeedsTable schema (default: false) - Create database migration 0063_add_import_tags_to_feeds.sql - Update zod schemas (zFeedSchema, zNewFeedSchema, zUpdateFeedSchema) to include importTags - Update Feed model to handle importTags in create and update methods - Update feedWorker to: - Read title and categories from RSS parser - Attach categories as tags to bookmarks when importTags is enabled - Log warnings if tag attachment fails Resolves #1996 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Mohamed Bassem <MohamedBassem@users.noreply.github.com> * feat(web): Add importTags option to feed settings UI - Add importTags toggle to FeedsEditorDialog (create feed) - Add importTags toggle to EditFeedDialog (edit feed) - Display as a bordered switch control with descriptive text - Defaults to false for new feeds Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com> * fix migration * remove extra migration --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com>
* feat: Make search job timeout configurableMohamed Bassem2025-11-021-1/+1
|
* fix: Stricter SSRF validation (#2082)Mohamed Bassem2025-11-027-98/+507
| | | | | | | | | | | | | | | | | | | * fix: Stricter SSRF validation * skip dns resolution if running in proxy context * more fixes * Add LRU cache * change the env variable for internal hostnames * make dns resolution timeout configerable * upgrade ipaddr * handle ipv6 * handle proxy bypass for request interceptor
* fix: More memory optimizations for crawler worker. #1748Mohamed Bassem2025-10-261-26/+43
|
* fix: fix screenshot filepath in crawlerMohamed Bassem2025-10-261-1/+1
|
* fix: Respect abort signal in admin maintenance jobsMohamed Bassem2025-10-263-1/+11
|
* deps: Upgrade metascraper pluginsMohamed Bassem2025-10-261-11/+11
|
* deps: Upgrade metascraper-readability 5.49.6Mohamed Bassem2025-10-261-1/+1
|
* feat: Allow configuring inline asset size thresholdMohamed Bassem2025-10-263-7/+5
|
* feat: Add admin maintenance job to migrate large inline HTML (#2071)Mohamed Bassem2025-10-264-4/+200
| | | | | | | * Add admin maintenance job to migrate large inline HTML * add cursor * more fixes
* refactor: generalize tidy assets queue into admin maintenance (#2059)Mohamed Bassem2025-10-264-112/+154
| | | | | * refactor: generalize admin maintenance queue * more fixes
* feat: Add source field to track bookmark creation sources (#2037)Mohamed Bassem2025-10-121-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * feat: Add source field to track bookmark creation sources Add a new 'source' field to the bookmarks table to track where bookmarks were created from. Possible values: api, web, cli, mobile, singlefile, rss. Changes: - Add source field to bookmarks table schema - Update Zod schemas to include source field - Update tRPC createBookmark procedure to store source - Update all callsites to pass appropriate source value: - api: Default to "api" if not provided - singlefile: Set to "singlefile" - rss: Set to "rss" in feedWorker - cli: Set to "cli" - mobile: Set to "mobile" in all mobile app bookmark creation - browser-extension: Set to "web" - web: Set to "web" in all web app bookmark creation - Create migration file for database schema change Fixes #2036 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Mohamed Bassem <MohamedBassem@users.noreply.github.com> * feat: Add extension source type for browser extension - Add 'extension' to bookmark source enum - Update browser extension to use 'extension' instead of 'web' Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com> * fix CI * fix CI * fix the migration file * add import source * make source nullish --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Mohamed Bassem <MohamedBassem@users.noreply.github.com>
* feat: support passing multiple proxy values (#2039)Mohamed Bassem2025-10-122-7/+14
| | | | | | | * feat: support passing multiple proxy values * fix typo * trim and filter
* fix: round feed refresh hour for idempotency (#2013)Mohamed Bassem2025-10-061-1/+6
|
* feat: Restate-based queue plugin (#2011)Mohamed Bassem2025-10-051-2/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WIP: Initial restate integration * add retry * add delay + idempotency * implement concurrency limits * add admin stats * add todos * add id provider * handle onComplete failures * add tests * add pub key and fix logging * add priorities * fail call after retries * more fixes * fix retries left * some refactoring * fix package.json * upgrade sdk * some test cleanups
* feat: use jpegs for screenshots instead of pngsMohamed Bassem2025-09-281-2/+3
|
* feat: Stop downloading video/audio in playwrightMohamed Bassem2025-09-281-0/+19
|
* fix: Abort dangling processing when crawler is aborted (#1988)Mohamed Bassem2025-09-281-27/+98
| | | | | | | | | | | * fix: Abort dangling processing when crawler is aborted * comments * report the size * handle unhandleded rejection * drop promisify
* fix: Cleanup temp assets on monolith timeoutMohamed Bassem2025-09-281-1/+17
|
* feat: Add tag search and pagination (#1987)Mohamed Bassem2025-09-281-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * feat: Add tag search and use in the homepage * use paginated query in the all tags view * wire the load more buttons * add skeleton to all tags page * fix attachedby aggregation * fix loading states * fix hasNextPage * use action buttons for load more buttons * migrate the tags auto complete to the search api * Migrate the tags editor to the new search API * Replace tag merging dialog with tag auto completion * Merge both search and list APIs * fix tags.list * add some tests for the endpoint * add relevance based sorting * change cursor * update the REST API * fix review comments * more fixes * fix lockfile * i18n * fix visible tags
* fix: fix bundling liteque in the workersMohamed Bassem2025-09-142-0/+2
|
* refactor: Move callsites to liteque to be behind a pluginMohamed Bassem2025-09-1413-123/+134
|
* feat: Add cookie support for browser page accessMohamed Bassem2025-09-071-0/+59
| | | | | | | | | | | * feat: Add cookie support for browser page access Implemented cookie functionality for browser page access, including BROWSER_COOKIE_PATH configuration to specify the cookies JSON file path. * fix the docs --------- Co-authored-by: lizz <lizong1204@gmail.com>
* feat(workers): add worker enable/disable lists (#1885)Mohamed Bassem2025-09-071-44/+49
|
* fix: fix assets being marked as pending summarizationMohamed Bassem2025-09-071-0/+7
|
* feat: add gif asset type support (#1876)Drashi2025-09-071-2/+8
| | | | | | | | | * feat: add gif asset type support * skip inference for gis --------- Co-authored-by: Mohamed Bassem <me@mbassem.com>
* fix: don't mark inferenace job as failed when there's no content. fixes #1666Mohamed Bassem2025-09-072-7/+32
|
* fix: fix pdf detection when the header contains charset. fix: #1677Mohamed Bassem2025-09-071-2/+16
|
* fix: Fix feed worker to fetch feeds with proxyMohamed Bassem2025-09-063-50/+58
|
* fix: Change the inferance working logging when disabled to be a debug log levelMohamed Bassem2025-09-062-2/+2
|
* fix: Dont attempt to fetch rss if the user if out of quotaMohamed Bassem2025-09-061-0/+13
|
* refactor: Extract quota logic into its own classMohamed Bassem2025-09-063-15/+13
|
* fix: Reduce polling interval on meilisearch tasksMohamed Bassem2025-09-061-1/+1
|
* fix: Don't enqueue video tasks when video downlaod is disabledMohamed Bassem2025-09-061-8/+10
|
* fix: fix long worker log lines when downloading base64 imagesMohamed Bassem2025-08-301-1/+3
|
* fix: Respect wal mode for the queue dbMohamed Bassem2025-08-301-1/+1
|
* fix: dangling assets created by changing crawling configMohamedBassem2025-08-221-5/+6
|
* fix(workers): Drop the withTimeout wrappersMohamedBassem2025-08-222-10/+2
|
* feat: Export prometheus metrics from the workersMohamedBassem2025-08-2214-5/+111
|
* refactor: Refactor crawlerWorker to use tryCatchMohamedBassem2025-07-271-123/+117
|
* refactor: Extract meilisearch as a pluginMohamedBassem2025-07-273-61/+45
|
* chore: More turbo fixesMohamedBassem2025-07-271-2/+2
|
* fix: Ensure that all packages are ESM packagesMohamedBassem2025-07-271-0/+1
|
* deps: Upgrade viteMohamed Bassem2025-07-261-1/+1
|
* fix: Run workers in prod without tsx. Fixes #1673Mohamed Bassem2025-07-192-2/+26
|
* feat: Allow setting browserless crawling per userMohamed Bassem2025-07-191-1/+19
|
* Revert "fix: Fix the types of the bookmark types in the db query"Mohamed Bassem2025-07-132-21/+1
| | | | This reverts commit 4ba3e8047a5b1f160169617187436c09e91662ec.
* fix: Fix the types of the bookmark types in the db queryMohamed Bassem2025-07-132-1/+21
|
* feat: Add proper proxy support. fixes #1265Mohamed Bassem2025-07-132-9/+87
|
* deps: Upgrade typescript to 5.8Mohamed Bassem2025-07-121-1/+1
|
generated by cgit v1.2.3-70-g09d2 (git 2.46.0) at 2026-02-07 02:48:40 +0000