| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | fix: round feed refresh hour for idempotency (#2013) | Mohamed Bassem | 2025-10-06 | 1 | -1/+6 |
| | | |||||
| * | feat: Restate-based queue plugin (#2011) | Mohamed Bassem | 2025-10-05 | 1 | -2/+8 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * WIP: Initial restate integration * add retry * add delay + idempotency * implement concurrency limits * add admin stats * add todos * add id provider * handle onComplete failures * add tests * add pub key and fix logging * add priorities * fail call after retries * more fixes * fix retries left * some refactoring * fix package.json * upgrade sdk * some test cleanups | ||||
| * | feat: use jpegs for screenshots instead of pngs | Mohamed Bassem | 2025-09-28 | 1 | -2/+3 |
| | | |||||
| * | feat: Stop downloading video/audio in playwright | Mohamed Bassem | 2025-09-28 | 1 | -0/+19 |
| | | |||||
| * | fix: Abort dangling processing when crawler is aborted (#1988) | Mohamed Bassem | 2025-09-28 | 1 | -27/+98 |
| | | | | | | | | | | | | * fix: Abort dangling processing when crawler is aborted * comments * report the size * handle unhandleded rejection * drop promisify | ||||
| * | fix: Cleanup temp assets on monolith timeout | Mohamed Bassem | 2025-09-28 | 1 | -1/+17 |
| | | |||||
| * | feat: Add tag search and pagination (#1987) | Mohamed Bassem | 2025-09-28 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * feat: Add tag search and use in the homepage * use paginated query in the all tags view * wire the load more buttons * add skeleton to all tags page * fix attachedby aggregation * fix loading states * fix hasNextPage * use action buttons for load more buttons * migrate the tags auto complete to the search api * Migrate the tags editor to the new search API * Replace tag merging dialog with tag auto completion * Merge both search and list APIs * fix tags.list * add some tests for the endpoint * add relevance based sorting * change cursor * update the REST API * fix review comments * more fixes * fix lockfile * i18n * fix visible tags | ||||
| * | fix: fix bundling liteque in the workers | Mohamed Bassem | 2025-09-14 | 2 | -0/+2 |
| | | |||||
| * | refactor: Move callsites to liteque to be behind a plugin | Mohamed Bassem | 2025-09-14 | 13 | -123/+134 |
| | | |||||
| * | feat: Add cookie support for browser page access | Mohamed Bassem | 2025-09-07 | 1 | -0/+59 |
| | | | | | | | | | | | | * feat: Add cookie support for browser page access Implemented cookie functionality for browser page access, including BROWSER_COOKIE_PATH configuration to specify the cookies JSON file path. * fix the docs --------- Co-authored-by: lizz <lizong1204@gmail.com> | ||||
| * | feat(workers): add worker enable/disable lists (#1885) | Mohamed Bassem | 2025-09-07 | 1 | -44/+49 |
| | | |||||
| * | fix: fix assets being marked as pending summarization | Mohamed Bassem | 2025-09-07 | 1 | -0/+7 |
| | | |||||
| * | feat: add gif asset type support (#1876) | Drashi | 2025-09-07 | 1 | -2/+8 |
| | | | | | | | | | | * feat: add gif asset type support * skip inference for gis --------- Co-authored-by: Mohamed Bassem <me@mbassem.com> | ||||
| * | fix: don't mark inferenace job as failed when there's no content. fixes #1666 | Mohamed Bassem | 2025-09-07 | 2 | -7/+32 |
| | | |||||
| * | fix: fix pdf detection when the header contains charset. fix: #1677 | Mohamed Bassem | 2025-09-07 | 1 | -2/+16 |
| | | |||||
| * | fix: Fix feed worker to fetch feeds with proxy | Mohamed Bassem | 2025-09-06 | 3 | -50/+58 |
| | | |||||
| * | fix: Change the inferance working logging when disabled to be a debug log level | Mohamed Bassem | 2025-09-06 | 2 | -2/+2 |
| | | |||||
| * | fix: Dont attempt to fetch rss if the user if out of quota | Mohamed Bassem | 2025-09-06 | 1 | -0/+13 |
| | | |||||
| * | refactor: Extract quota logic into its own class | Mohamed Bassem | 2025-09-06 | 3 | -15/+13 |
| | | |||||
| * | fix: Reduce polling interval on meilisearch tasks | Mohamed Bassem | 2025-09-06 | 1 | -1/+1 |
| | | |||||
| * | fix: Don't enqueue video tasks when video downlaod is disabled | Mohamed Bassem | 2025-09-06 | 1 | -8/+10 |
| | | |||||
| * | fix: fix long worker log lines when downloading base64 images | Mohamed Bassem | 2025-08-30 | 1 | -1/+3 |
| | | |||||
| * | fix: Respect wal mode for the queue db | Mohamed Bassem | 2025-08-30 | 1 | -1/+1 |
| | | |||||
| * | fix: dangling assets created by changing crawling config | MohamedBassem | 2025-08-22 | 1 | -5/+6 |
| | | |||||
| * | fix(workers): Drop the withTimeout wrappers | MohamedBassem | 2025-08-22 | 2 | -10/+2 |
| | | |||||
| * | feat: Export prometheus metrics from the workers | MohamedBassem | 2025-08-22 | 14 | -5/+111 |
| | | |||||
| * | refactor: Refactor crawlerWorker to use tryCatch | MohamedBassem | 2025-07-27 | 1 | -123/+117 |
| | | |||||
| * | refactor: Extract meilisearch as a plugin | MohamedBassem | 2025-07-27 | 3 | -61/+45 |
| | | |||||
| * | chore: More turbo fixes | MohamedBassem | 2025-07-27 | 1 | -2/+2 |
| | | |||||
| * | fix: Ensure that all packages are ESM packages | MohamedBassem | 2025-07-27 | 1 | -0/+1 |
| | | |||||
| * | deps: Upgrade vite | Mohamed Bassem | 2025-07-26 | 1 | -1/+1 |
| | | |||||
| * | fix: Run workers in prod without tsx. Fixes #1673 | Mohamed Bassem | 2025-07-19 | 2 | -2/+26 |
| | | |||||
| * | feat: Allow setting browserless crawling per user | Mohamed Bassem | 2025-07-19 | 1 | -1/+19 |
| | | |||||
| * | Revert "fix: Fix the types of the bookmark types in the db query" | Mohamed Bassem | 2025-07-13 | 2 | -21/+1 |
| | | | | | This reverts commit 4ba3e8047a5b1f160169617187436c09e91662ec. | ||||
| * | fix: Fix the types of the bookmark types in the db query | Mohamed Bassem | 2025-07-13 | 2 | -1/+21 |
| | | |||||
| * | feat: Add proper proxy support. fixes #1265 | Mohamed Bassem | 2025-07-13 | 2 | -9/+87 |
| | | |||||
| * | deps: Upgrade typescript to 5.8 | Mohamed Bassem | 2025-07-12 | 1 | -1/+1 |
| | | |||||
| * | deps: Upgrade drizzle | Mohamed Bassem | 2025-07-12 | 1 | -1/+1 |
| | | |||||
| * | fix: Prioritize crawling user added links over bulk imports. fixes #1717 | Mohamed Bassem | 2025-07-12 | 5 | -24/+55 |
| | | |||||
| * | fix: Fix search indexing after content split | Mohamed Bassem | 2025-07-06 | 1 | -7/+4 |
| | | |||||
| * | feat: Store large html content in the asset db | Mohamed Bassem | 2025-07-06 | 5 | -9/+135 |
| | | |||||
| * | feat: Add per user storage quota | Mohamed Bassem | 2025-07-06 | 4 | -75/+183 |
| | | |||||
| * | feat(workers): Allow custmoizing max parallelism for a bunch of workers. ↵ | Mohamed Bassem | 2025-07-05 | 5 | -5/+7 |
| | | | | | Fixes #724 | ||||
| * | fix(workers): A more lenient JSON parsing for LLM responses. Fixes #1267 | Mohamed Bassem | 2025-07-04 | 1 | -1/+39 |
| | | |||||
| * | fix(workers): Disable the metascraper readability as it's causing slowness ↵ | Mohamed Bassem | 2025-06-22 | 1 | -2/+0 |
| | | | | | in worker | ||||
| * | fix(workers): Fix jsdom console logs leaking into worker logs | Mohamed Bassem | 2025-06-22 | 1 | -2/+3 |
| | | |||||
| * | feat(workers): adding a local metascraper plugin for Reddit posts (#1302) | David Woods | 2025-06-22 | 3 | -13/+115 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * chore: metascraper 5.x comes with its own types, including @types/metascraper is now redundant; also updating to latest versions of metascraper libraries * feat (workers): creating a local metascraper plugin for Reddit posts In the past, the preview images for bookmarks from Reddit links were poorly chosen. Reddit does not use opengraph tags, so metascraper-images simply looked for all images on the page and returned the first. This tended to be the profile picture for the poster for the Reddit link. This new plugin, using the existing metascraper framework, provides a better selection of image for the bookmark when the URL domain is 'reddit'. In addition, recent changes (I believe this was a side effect of adding the metascraper-author and/or the metascaper-publisher plugins, but it could also be related to the metascraper-readibility plugin) broke what used to be a good choice of bookmark title. Previously, titles looked like 'Tinyauth just reached 1000 stars! : r/selfhosted' with both thread title and subreddit mentioned. After this update, all Reddit posts now have the same title: 'The heart of the internet'. To return to the better format, this new metascraper-reddit plugin now attempts to retrieve the better title from reddit URLs. Note that in order to gain precendence in title selection, the 'metascraperReddit()' inclusion in the crawlerWorkers.ts metascraper instantiation list had to be moved above metascraperReadability(). * chore: updated Hoarder in text to Karakeep * chore: update metascraper versions fix for metascraper types has been merged; the expect-error comment can be removed * chore: merge with master --------- Co-authored-by: Mohamed Bassem <me@mbassem.com> | ||||
| * | feat(workers): migrate from puppeteer to playwright (#1296) | Mael | 2025-06-22 | 2 | -34/+39 |
| | | | | | | | | | | | | | | | | | | | | | | | | * feat: convert to playwright Convert crawling to use Playwright instead of Chrome. - Update Dockerfile to include Playwright - Update crawler worker to use Playwright API - Update dependencies * feat: convert from Puppeteer to Playwright for crawling * feat: update docker-compose * use separate browser context for better isolation * skip chrome download in linux script * readd the stealth plugin --------- Co-authored-by: Mohamed Bassem <me@mbassem.com> | ||||
| * | chore: More oxlint changes | Mohamed Bassem | 2025-06-22 | 3 | -7/+4 |
| | | |||||
| * | chore: migrate away from eslint to oxlint (#1642) | xuatz | 2025-06-22 | 5 | -12/+27 |
| | | | | | | | | * chore: migrate away from eslint to oxlint * revert turbo task name lint * it seems like we can remove the seemingly default globals | ||||
 |
index : karakeep | |
| Unnamed repository; edit this file 'description' to name the repository. |
| aboutsummaryrefslogtreecommitdiffstats |