| 2024-05-04 10:29 | exclude and shutdown notice | René Wagner | 2 | +4 | -0 |
| 2024-04-25 10:42 | more excludes and a news | René Wagner | 3 | +12 | -2 |
| 2024-03-10 12:47 | new mail adress | René Wagner | 2 | +5 | -4 |
| 2024-01-25 18:07 | birthday news | René Wagner | 3 | +16 | -9 |
| 2023-12-23 12:51 | include ipv6 information | René Wagner | 2 | +4 | -0 |
| 2023-11-13 07:09 | new exclude | René Wagner | 1 | +1 | -0 |
| 2023-10-16 17:25 | switch back to release version of jetforce | René Wagner | 3 | +5 | -3 |
| 2023-09-18 08:49 | switch links to geminiprotocol.net | René Wagner | 4 | +6 | -9 |
| 2023-09-16 11:59 | allow robots.txt served as any text/* mimetype | René Wagner | 2 | +11 | -2 |
| 2023-08-22 08:14 | new exclude and typo-fix on robots.txt usage doc | René Wagner | 3 | +45 | -12 |
| 2023-08-12 18:57 | docs: robots.txt clarification | René Wagner | 2 | +8 | -3 |
| 2023-08-05 19:12 | new excludes and README update | René Wagner | 3 | +12 | -11 |
| 2023-07-30 15:07 | pubnix-aware fetching of robots.txt | René Wagner | 5 | +38 | -30 |
| 2023-07-30 14:33 | make 'newest hosts' URI absolute including scheme | René Wagner | 1 | +1 | -1 |
| 2023-07-30 10:07 | rework fetchable_url generation | René Wagner | 3 | +28 | -114 |
| 2023-07-29 18:55 | fix typo in robots.txt | René Wagner | 1 | +1 | -1 |
| 2023-07-29 08:12 | update news | René Wagner | 3 | +7 | -1 |
| 2023-07-08 18:42 | new excludes, fixes for infra scripts | René Wagner | 4 | +23 | -6 |
| 2023-06-08 14:50 | fix backlinks search | René Wagner | 1 | +2 | -2 |
| 2023-06-07 18:02 | further simplify URI handling and always use lowercase host part | René Wagner | 11 | +70 | -110 |
| 2023-06-04 17:41 | fix some issues when deleting outdated pages | René Wagner | 1 | +18 | -21 |
| 2023-06-04 17:41 | revert changes made for search.clttr.info | René Wagner | 4 | +15 | -29 |
| 2023-06-03 12:12 | fix search | René Wagner | 1 | +21 | -23 |
| 2023-06-03 12:07 | list filtered URIs in sorted manner | René Wagner | 1 | +1 | -1 |
| 2023-06-03 12:06 | remove unused dependency 'twisted' | René Wagner | 2 | +2 | -16 |
| 2023-06-03 11:45 | update pyproject.toml | René Wagner | 1 | +2 | -2 |
| 2023-06-03 11:31 | fix deprecation warnings | René Wagner | 1 | +11 | -11 |
| 2023-06-03 08:14 | fix handling of requests without path segment | René Wagner | 3 | +15 | -12 |
| 2023-06-02 11:51 | adjust params to new vps | René Wagner | 3 | +6 | -3 |
| 2023-04-25 15:53 | update about due to new donation | René Wagner | 1 | +1 | -1 |
| 2023-04-18 18:07 | fix page deletion | René Wagner | 1 | +3 | -5 |
| 2023-04-14 10:57 | donation info | René Wagner | 7 | +23 | -22 |
| 2023-04-09 09:56 | pull whoosh directly from repo, preparations to move to search.clttr.info | René Wagner | 5 | +7 | -10 |
| 2023-04-06 18:35 | param tweaks for whoosh and sqlite | Rene Wagner | 3 | +4 | -3 |
| 2023-03-30 18:47 | shell script cosmetics, switch to jetforce master branch | Rene Wagner | 5 | +23 | -20 |
| 2023-03-12 11:33 | fix page diff evaluation on recrawl | Rene Wagner | 4 | +11 | -827 |
| 2023-03-09 11:32 | fix storing of page content | Rene Wagner | 1 | +19 | -18 |
| 2023-03-08 20:31 | split content into separate table | Rene Wagner | 3 | +38 | -10 |
| 2023-03-08 19:02 | update README | Rene Wagner | 4 | +44 | -29 |
| 2023-03-05 15:40 | include twtxt.txt feeds in known-feeds page | René Wagner | 2 | +7 | -0 |
| 2023-02-11 09:37 | show list of excluded uris | René Wagner | 4 | +31 | -1 |
| 2023-02-10 07:44 | excludes, cosmetics, infra fixes | René Wagner | 5 | +55 | -40 |
| 2023-02-05 15:02 | fix crawl.lock handling | René Wagner | 2 | +18 | -1 |
| 2023-01-29 08:48 | update project url for gusmobile and logging settings | René Wagner | 3 | +46 | -134 |
| 2023-01-27 14:20 | new excludes & news | René Wagner | 3 | +12 | -2 |
| 2023-01-26 20:51 | add new exclude & test | René Wagner | 2 | +19 | -0 |
| 2023-01-06 10:55 | fix crawl, revamp crawl & indexing procedure | René Wagner | 6 | +16 | -33 |
| 2023-01-06 09:51 | update logging settings | René Wagner | 1 | +8 | -2 |
| 2023-01-06 09:09 | update db schema for improved query performance | René Wagner | 5 | +16 | -16 |
| 2023-01-05 19:57 | optimize db index for faster serve startup | René Wagner | 1 | +1 | -1 |
| 2023-01-01 18:35 | news 2023-01-01 | René Wagner | 2 | +13 | -4 |
| 2022-12-09 20:42 | remove invalid uris from seed-requests.txt | René Wagner | 1 | +1 | -1 |
| 2022-12-09 20:19 | only use whoosh multisegments when indexing from scratch | René Wagner | 1 | +3 | -2 |
| 2022-12-09 14:56 | update package versions | René Wagner | 1 | +62 | -157 |
| 2022-12-09 14:52 | use incremental index update as default | René Wagner | 2 | +8 | -14 |
| 2022-11-20 11:10 | new exclude, make crawl usw 3 threads | René Wagner | 3 | +7 | -3 |
| 2022-11-20 11:09 | news 2022-11-19 | René Wagner | 2 | +11 | -0 |
| 2022-10-26 17:37 | fix index rebuild on first day of month | René Wagner | 1 | +1 | -1 |
| 2022-10-26 17:37 | increase memory for indexer | René Wagner | 1 | +1 | -1 |
| 2022-10-26 17:36 | exclude webgate.geminet.org | René Wagner | 1 | +1 | -0 |
| 2022-08-25 07:30 | fix link in latest news | René Wagner | 1 | +1 | -1 |
| 2022-08-23 14:47 | properly implement deletion of capsules with outdated crawls | René Wagner | 2 | +28 | -18 |
| 2022-08-22 15:25 | add donation link | René Wagner | 2 | +10 | -0 |
| 2022-08-18 08:57 | news 2022-08-18 | René Wagner | 1 | +6 | -0 |
| 2022-08-16 17:38 | fix normalizing of URIs with default port | René Wagner | 2 | +7 | -9 |
| 2022-08-14 15:35 | fix test and add additional test for special robots.txt | René Wagner | 3 | +16 | -3 |
| 2022-08-06 13:26 | fix -d param for crawl | René Wagner | 1 | +1 | -1 |
| 2022-08-06 13:10 | update deps | René Wagner | 1 | +17 | -241 |
| 2022-08-06 13:04 | Merge pull request 'Upgrade feedparser to 6.0.10' (#51) from duncan-bayne/geminispace.info:master into master | René W | 2 | +258 | -29 |
| 2022-08-06 10:37 | Upgrade feedparser to 6.0.10 | Duncan Bayne | 2 | +258 | -29 |
| 2022-07-23 06:40 | newest hosts template | René Wagner | 1 | +1 | -1 |
| 2022-07-23 06:37 | show 50 newest hosts instead of 30 | René Wagner | 1 | +1 | -1 |
| 2022-07-23 06:36 | exclude auragem.space/twitch/ | René Wagner | 5 | +50 | -243 |
| 2022-06-10 17:54 | disable search suggestions due to bug | René Wagner | 3 | +28 | -27 |
| 2022-05-26 17:42 | move data deletion to indexing | René Wagner | 3 | +41 | -36 |
| 2022-05-16 17:01 | news 2021-05-16 | René Wagner | 5 | +8 | -67 |
| 2022-05-13 15:39 | make crawl multi-threaded | René Wagner | 2 | +42 | -26 |
| 2022-05-09 19:45 | disable mmap-ing of whoosh index | René Wagner | 1 | +1 | -1 |
| 2022-05-08 17:29 | switch SQLite to WAL mode | René Wagner | 5 | +21 | -9 |
| 2022-05-08 09:03 | some adjustments | René Wagner | 2 | +11 | -4 |
| 2022-05-08 08:20 | updated dependencies, excludes | René Wagner | 4 | +217 | -203 |
| 2022-03-25 20:08 | news 2022-03-25 | René Wagner | 1 | +3 | -0 |
| 2022-03-23 07:38 | tweak whoosh writer settings for speedup | René Wagner | 1 | +1 | -2 |
| 2022-03-20 08:18 | workaround breaking change in markupsafe 2.1.x | René Wagner | 4 | +43 | -43 |
| 2022-03-19 17:55 | news 2022-03-19 | René Wagner | 2 | +10 | -2 |
| 2022-03-19 17:54 | update deps | René Wagner | 2 | +162 | -156 |
| 2022-03-03 18:34 | fix some typos | René Wagner | 2 | +1 | -2 |
| 2022-02-18 17:37 | add info about redirect indexing | René Wagner | 3 | +10 | -9 |
| 2022-02-06 18:53 | add index for speedup | René Wagner | 3 | +17 | -5 |
| 2022-02-06 18:13 | some sql adjustments | René Wagner | 3 | +17 | -13 |
| 2022-02-05 15:17 | precompute feeds and pages | René Wagner | 2 | +27 | -33 |
| 2022-02-05 09:37 | precompute hosts statistics | René Wagner | 2 | +26 | -30 |
| 2022-02-05 09:17 | update deps | René Wagner | 1 | +134 | -130 |
| 2022-02-05 09:17 | exclude git.skyjake.fi | René Wagner | 1 | +3 | -0 |
| 2022-02-04 18:25 | exclude tlgs.one | René Wagner | 1 | +1 | -0 |
| 2022-02-04 08:41 | generic exception handling for page crawling | René Wagner | 1 | +26 | -20 |
| 2022-01-28 12:12 | new exclude: taz.de | René Wagner | 1 | +1 | -0 |
| 2022-01-25 20:11 | news 2022-01-25 | René Wagner | 1 | +4 | -0 |
| 2022-01-04 18:57 | news 2021-12-29 | René Wagner | 1 | +4 | -0 |
| 2021-12-29 09:59 | don't delete excluded pages from the pages table | René Wagner | 1 | +0 | -10 |
| 2021-12-29 09:57 | update poetry version | René Wagner | 1 | +170 | -148 |
| 2021-11-24 19:45 | show 30 latest hosts | René Wagner | 2 | +4 | -1 |
| 2021-11-20 16:13 | exclude antenna filters | René Wagner | 1 | +3 | -0 |
| 2021-11-19 15:06 | don't crash on URIs with non-number port | René Wagner | 1 | +1 | -1 |
| 2021-11-16 15:21 | update excludes | René Wagner | 1 | +5 | -2 |
| 2021-11-11 17:28 | update contact | René Wagner | 3 | +3 | -3 |
| 2021-11-09 17:41 | dependency update | René Wagner | 1 | +76 | -74 |
| 2021-11-07 16:27 | cleanup excludes | René Wagner | 1 | +1 | -2 |
| 2021-10-25 18:45 | save first_seen_at if a page is created through a link | René Wagner | 3 | +172 | -150 |
| 2021-10-14 18:22 | add link to source in geminispace | René Wagner | 1 | +2 | -1 |
| 2021-10-14 16:54 | more meta data for index cleanup | René Wagner | 6 | +24 | -40 |
| 2021-10-11 18:03 | avoid crash when normalized_url is not set | René Wagner | 1 | +16 | -11 |
| 2021-10-11 17:45 | use cronjob for automated start | René Wagner | 4 | +3 | -24 |
| 2021-09-16 17:53 | some cleanup | René Wagner | 5 | +10 | -35 |
| 2021-09-06 06:19 | fix broken link to source code | René Wagner | 1 | +4 | -3 |
| 2021-09-04 07:03 | do not add every single domain to the statistics file | René Wagner | 5 | +18 | -11 |
| 2021-08-18 15:23 | news 2021-08-18 | René Wagner | 2 | +4 | -1 |
| 2021-08-17 19:00 | some minor changes | René Wagner | 4 | +33 | -57 |
| 2021-08-10 16:43 | ensure that scheme is given when searching for backlinks | René Wagner | 1 | +2 | -0 |
| 2021-08-10 16:37 | update 2021-08-07 | René Wagner | 2 | +7 | -0 |
| 2021-08-06 14:50 | ensure that seed-requests use absolute URIs | René Wagner | 2 | +3 | -0 |
| 2021-08-06 14:41 | more excludes | René Wagner | 2 | +45 | -53 |
| 2021-07-23 11:11 | implemented deletion of outdated data | René Wagner | 1 | +17 | -1 |
| 2021-07-20 17:14 | small fixes and doc adjustments | René Wagner | 4 | +15 | -10 |
| 2021-07-17 17:40 | remove obsolete code | René Wagner | 4 | +1 | -123 |
| 2021-07-17 09:06 | support prioritized robots.txt user-agents | Hannu Hartikainen | 3 | +102 | -5 |
| 2021-07-17 10:35 | more excludes and less logging | René Wagner | 2 | +3 | -1 |
| 2021-07-14 19:01 | treat schemeless links as non-gemini links | René Wagner | 1 | +3 | -4 |
| 2021-07-14 18:56 | remove pikkulog separation | René Wagner | 2 | +0 | -20 |
| 2021-07-14 06:36 | minor code cleanup in db_model | René Wagner | 3 | +4 | -19 |
| 2021-07-14 06:32 | update to some templates | René Wagner | 5 | +5 | -16 |
| 2021-07-13 15:20 | remove Search model | René Wagner | 3 | +4 | -16 |
| 2021-07-13 11:21 | enable 'newest-hosts' and 'newest-pages' sites again | René Wagner | 7 | +50 | -13 |
| 2021-07-13 07:21 | remove raw data from excluded capsules | René Wagner | 2 | +10 | -1 |
| 2021-07-12 19:37 | index text files up to 5 MB | René Wagner | 4 | +25 | -21 |
| 2021-07-12 17:27 | commit search index only when indexing is complete | René Wagner | 4 | +26 | -124 |
| 2021-07-12 14:57 | store document id in whoosh index | René Wagner | 1 | +1 | -1 |
| 2021-07-12 12:58 | some tweaks to indexing | René Wagner | 3 | +6 | -6 |
| 2021-07-11 17:03 | restructure crawl data | René Wagner | 17 | +88 | -165 |
| 2021-07-11 07:05 | remove Crawl table, all info is stored in page table now | René Wagner | 5 | +84 | -157 |
| 2021-07-10 07:08 | don't persist robots.txt over multiple crawls | René Wagner | 2 | +3 | -24 |
| 2021-07-09 20:05 | improve indexing speed via optimized backlinks query | René Wagner | 3 | +4 | -17 |
| 2021-07-09 15:38 | again a new exclude | René Wagner | 1 | +1 | -0 |
| 2021-07-09 15:37 | move gusmobile to new home | René Wagner | 2 | +115 | -110 |
| 2021-07-04 19:49 | update 2021-07-04 & more excludes | René Wagner | 2 | +24 | -7 |
| 2021-06-28 07:31 | additional filter | René Wagner | 2 | +2 | -1 |
| 2021-06-26 11:16 | update 2021-06-26 | René Wagner | 2 | +8 | -1 |
| 2021-06-16 19:18 | exclude godocs.io | René Wagner | 2 | +6 | -0 |
| 2021-06-14 07:13 | error handling on page crawl save | René Wagner | 1 | +29 | -7 |
| 2021-06-04 09:40 | update 2021-06-04 | René Wagner | 1 | +9 | -0 |
| 2021-05-29 08:56 | more exception handling on link update | René Wagner | 1 | +4 | -2 |
| 2021-05-27 13:24 | fix wrong embedding of excludes | René Wagner | 2 | +8 | -4 |
| 2021-05-26 11:06 | unify capitalisation of charset in statistics | René Wagner | 1 | +2 | -2 |
| 2021-05-25 20:05 | move exclude definition to own file | René Wagner | 5 | +253 | -251 |
| 2021-05-25 19:13 | news 2021-05-25 | René Wagner | 2 | +5 | -0 |
| 2021-05-21 19:58 | some exception handling and updated service files | René Wagner | 4 | +11 | -8 |
| 2021-05-16 07:59 | fix last wrong exception in crawl | René Wagner | 2 | +1 | -2 |
| 2021-05-14 18:59 | fix wrong exception handling in crawl | René Wagner | 2 | +107 | -116 |
| 2021-05-12 15:46 | update 2021-05-12 | René Wagner | 2 | +7 | -2 |
| 2021-05-10 15:41 | rewrite statistics gathering to pure sql | René Wagner | 3 | +28 | -27 |
| 2021-05-08 19:51 | exception handling on page save | René Wagner | 4 | +279 | -223 |
| 2021-04-14 19:33 | news 2021-04-14 | René Wagner | 1 | +4 | -0 |
| 2021-04-05 06:07 | delete tmp files of whoosh | René Wagner | 1 | +1 | -0 |
| 2021-03-25 20:33 | use .fromisoformat for getting timestamp from db | René Wagner | 1 | +1 | -1 |
| 2021-03-25 20:10 | various corrections | René Wagner | 3 | +6 | -3 |
| 2021-03-20 19:58 | hack: index update in separate dir | René Wagner | 5 | +13 | -13 |
| 2021-03-08 18:21 | skip a capsule after 5 consecutive failed requests | René Wagner | 2 | +28 | -11 |
| 2021-03-08 17:59 | workaround for "index update blocks searches" | René Wagner | 2 | +8 | -1 |
| 2021-03-08 17:59 | news update 2021-03-08 | René Wagner | 1 | +8 | -1 |
| 2021-03-08 17:51 | Merge branch 'master' of git://natpen.net/gus | René Wagner | 1 | +2 | -0 |
| 2021-03-05 18:02 | update poetry deps | René Wagner | 1 | +113 | -120 |
| 2021-02-26 17:52 | gsi specific updates 2021-02-26 | René Wagner | 2 | +6 | -1 |
| 2021-02-22 18:06 | robots.txt sections "*" and "indexer" are honored | René Wagner | 2 | +4 | -13 |
| 2021-02-12 07:05 | correctly handle robots.txt | René Wagner | 2 | +26 | -8 |
| 2021-02-12 07:53 | add verbose search to robots.txt | René Wagner | 1 | +1 | -0 |
| 2021-02-12 07:53 | add verbose search to robots.txt | René Wagner | 1 | +1 | -0 |
| 2021-02-12 07:05 | correctly handle robots.txt | René Wagner | 2 | +26 | -8 |
| 2021-02-10 18:05 | Merge branch 'master' of git://natpen.net/gus | René Wagner | 1 | +3 | -3 |
| 2021-02-10 10:06 | limit max_crawl_depth to 100 for normal crawl | René Wagner | 1 | +1 | -1 |
| 2021-02-10 06:07 | increase frequency to avoid rescanning within a single crawl | René Wagner | 1 | +3 | -3 |
| 2021-02-08 16:43 | add some forbidden URIs & set max_crawl_depth | René Wagner | 1 | +38 | -22 |
| 2021-02-07 18:11 | remove seed-requests from repo | René Wagner | 2 | +5 | -97 |
| 2021-02-07 16:48 | Merge branch 'master' of git://natpen.net/gus | René Wagner | 2 | +10 | -1 |
| 2021-02-07 16:23 | Add a few more url parsing test cases | Natalie Pendragon | 1 | +3 | -0 |
| 2021-02-07 16:20 | Update to Python 3.9 compatibility | Natalie Pendragon | 1 | +7 | -1 |
| 2021-02-04 20:06 | update python deps | René Wagner | 1 | +293 | -278 |
| 2021-02-04 20:05 | introduce systemd-unit for indexer | René Wagner | 4 | +19 | -12 |
| 2021-02-02 17:38 | update python deps | René Wagner | 1 | +293 | -278 |
| 2021-02-02 16:39 | updates geminispace.info 2021-02-02 | René Wagner | 4 | +11 | -3 |
| 2021-01-31 20:08 | introduce systemd-unit for indexer | René Wagner | 3 | +17 | -4 |
| 2021-01-31 14:04 | gsi specific updates | René Wagner | 2 | +5 | -2 |
| 2021-01-30 15:15 | Make README heading lines more consistent | Natalie Pendragon | 1 | +5 | -5 |
| 2021-01-30 15:05 | Fix trailing whitespace and reformat long string | Natalie Pendragon | 1 | +10 | -2 |
| 2021-01-30 15:15 | Make README heading lines more consistent | Natalie Pendragon | 1 | +5 | -5 |
| 2021-01-29 09:08 | add systemd-units for automatic crawling | René Wagner | 3 | +46 | -10 |
| 2021-01-30 15:05 | Fix trailing whitespace and reformat long string | Natalie Pendragon | 1 | +10 | -2 |
| 2021-01-28 10:33 | add "/robots.txt" route to views.py | René Wagner | 1 | +4 | -0 |
| 2021-01-29 13:43 | gsi specific updates 2021-01-29 | René Wagner | 4 | +11 | -10 |
| 2021-01-28 19:59 | add systemd-units for automatic crawling | René Wagner | 3 | +46 | -10 |
| 2021-01-27 12:35 | add "/robots.txt" route to views.py | René Wagner | 1 | +4 | -0 |
| 2021-01-27 09:23 | modify views to match geminispace.info | René Wagner | 9 | +29 | -93 |
| 2021-01-21 20:08 | add seeds & update ignored urls | Gogs | 2 | +119 | -2 |
| 2020-12-26 17:30 | Defer search requests to threads | ugla | 1 | +38 | -30 |
| 2020-12-22 11:46 | Health test script and systemd service | Remco | 2 | +49 | -0 |
| 2020-12-22 15:00 | [serve] Fix copy-paste error in status endpoint function name | Natalie Pendragon | 1 | +1 | -1 |
| 2020-12-21 17:04 | [serve] Add status endpoint | Natalie Pendragon | 1 | +5 | -0 |
| 2020-12-08 15:10 | [serve] Improve formatting of statistics page | Natalie Pendragon | 1 | +4 | -4 |
| 2020-12-06 16:29 | [build_index] Import should_skip | Natalie Pendragon | 1 | +1 | -1 |
| 2020-12-06 16:28 | Refactor change frequency constants | Natalie Pendragon | 2 | +39 | -24 |
| 2020-12-05 14:04 | [crawl] Abort robots.txt parsing attempt if not text/plain | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-26 19:56 | [serve] Update contributions list on about page | Natalie Pendragon | 1 | +4 | -2 |
| 2020-11-26 19:47 | Bind to both IPv4 and IPv6 | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-23 02:50 | [crawl] Ignore another radio stream | Natalie Pendragon | 1 | +2 | -1 |
| 2020-11-20 22:37 | Speed up get_newest_hosts | Remco | 1 | +9 | -10 |
| 2020-11-17 14:09 | Add some more tests of GeminiResource | Natalie Pendragon | 1 | +30 | -0 |
| 2020-11-17 13:32 | Add regex-based url exclusion support & refactor tests | Natalie Pendragon | 5 | +83 | -42 |
| 2020-11-16 13:50 | Add TODO to README | Natalie Pendragon | 1 | +1 | -0 |
| 2020-11-16 13:44 | Take exclusions into account when generating statistics | Natalie Pendragon | 1 | +10 | -5 |
| 2020-11-16 13:01 | [serve] Fix formatting of dates on statistics page | Natalie Pendragon | 1 | +3 | -3 |
| 2020-11-16 12:50 | Add two new TODOs to README | Natalie Pendragon | 1 | +2 | -0 |
| 2020-11-16 12:49 | [build_index] Only index text pages <= 1KB in size | Natalie Pendragon | 3 | +5 | -2 |
| 2020-11-16 12:49 | More exclusions | Natalie Pendragon | 1 | +6 | -0 |
| 2020-11-16 12:47 | [serve] Fix index closing when program is killed | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-15 15:56 | [crawl] Increase increment to temp error change frequency | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-15 14:19 | [serve] Update indexing documentation | Natalie Pendragon | 1 | +8 | -0 |
| 2020-11-15 13:41 | [serve] Update about page | Natalie Pendragon | 1 | +5 | -3 |
| 2020-11-15 13:30 | Bump rolling writer's batch size back up to 5000 | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-15 13:30 | More exclusions | Natalie Pendragon | 1 | +8 | -0 |
| 2020-11-14 16:06 | Add systemd config | Natalie Pendragon | 1 | +22 | -0 |
| 2020-11-13 13:24 | Move all whoosh related stuff into separate module | Remco | 5 | +165 | -169 |
| 2020-11-12 20:03 | A friend for the other duck | Remco | 1 | +4 | -0 |
| 2020-11-11 12:27 | Bump dependencies | Natalie Pendragon | 1 | +163 | -129 |
| 2020-11-11 12:18 | [build_index] Fix logging statement | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-11 12:17 | [serve] Add statistics_overall_historical template | Natalie Pendragon | 1 | +14 | -0 |
| 2020-11-06 13:56 | Add .git-blame-ignore-revs file | Natalie Pendragon | 1 | +2 | -0 |
| 2020-11-06 13:44 | [crawl] Make logging message slightly clearer | Natalie Pendragon | 1 | +1 | -1 |
| 2020-11-06 13:44 | Check for null input in new strip_control_chars function | Natalie Pendragon | 1 | +2 | -0 |
| 2020-11-06 13:43 | Update default logging config to log to both console and file | Natalie Pendragon | 1 | +11 | -5 |
| 2020-11-06 13:42 | Reformat code with Black | Natalie Pendragon | 14 | +685 | -404 |
| 2020-11-06 12:22 | [crawl] Strip control chars from URLs in crawl logging | Natalie Pendragon | 1 | +46 | -29 |
| 2020-11-03 13:38 | Add exclusion improvement TODO to README | Natalie Pendragon | 1 | +1 | -0 |
| 2020-11-01 14:39 | Ignore link like lines in preformatted text blocks | Remco van 't Veer | 3 | +46 | -2 |
| 2020-11-02 13:39 | Add contributors section to about page | Natalie Pendragon | 1 | +20 | -0 |
| 2020-11-02 13:38 | Fix the index build | Natalie Pendragon | 3 | +34 | -18 |
| 2020-11-01 16:05 | Clean up todo list in README | Natalie Pendragon | 1 | +7 | -22 |
| 2020-10-31 14:06 | [build_index] Flush index segments to disk periodically | Natalie Pendragon | 1 | +15 | -3 |
| 2020-10-31 15:53 | Logging | Remco van 't Veer | 5 | +144 | -86 |
| 2020-10-31 15:53 | Drop unused imports | Remco van 't Veer | 3 | +12 | -52 |
| 2020-10-31 11:23 | Update gusmobile clone location in pyproject.toml | Natalie Pendragon | 1 | +1 | -1 |
| 2020-10-27 19:26 | Include notes on updating the index | Remco van 't Veer | 1 | +3 | -1 |
| 2020-10-27 16:02 | Describe procedure to get gus up and running | Remco van 't Veer | 1 | +30 | -0 |
| 2020-10-27 16:02 | Fix missing database column indexed_at on Page | Remco van 't Veer | 1 | +1 | -0 |
| 2020-10-28 10:55 | [crawl] Add a few new exclusions | Natalie Pendragon | 1 | +17 | -0 |
| 2020-10-28 10:50 | [build_index] Perform prefix-based URL exclusion during index build | Natalie Pendragon | 1 | +8 | -0 |
| 2020-09-16 12:56 | [serve] Add "jump to page" functionality to search | Natalie Pendragon | 2 | +18 | -0 |
| 2020-09-16 12:43 | [serve] Upgrade to Jetforce v0.6.0 | Natalie Pendragon | 3 | +628 | -196 |
| 2020-09-16 11:02 | [serve] Add more quotes | Natalie Pendragon | 1 | +17 | -0 |
| 2020-09-06 10:21 | [serve] Update documentation and links a bit | Natalie Pendragon | 4 | +15 | -8 |
| 2020-09-04 12:21 | [serve] Add dynamic quotes to footer | Natalie Pendragon | 3 | +66 | -17 |
| 2020-09-04 11:50 | [serve] Add newest pages endpoint, revamp documentation and index | Natalie Pendragon | 11 | +181 | -68 |
| 2020-09-03 12:00 | [serve] Add newest hosts route | Natalie Pendragon | 4 | +36 | -0 |
| 2020-08-25 08:37 | [serve] Remove extra quotation mark in add seeds template | Natalie Pendragon | 1 | +1 | -1 |
| 2020-08-11 12:30 | [crawl] Print change_frequency | Natalie Pendragon | 1 | +2 | -2 |
| 2020-08-11 12:18 | Fix bug in GeminiResource url construction | Natalie Pendragon | 1 | +3 | -3 |
| 2020-08-09 13:18 | [threads] Only work with textual pages | Natalie Pendragon | 1 | +3 | -0 |
| 2020-08-05 18:33 | [serve] Add favicon.txt route | Natalie Pendragon | 1 | +5 | -0 |
| 2020-08-05 13:03 | [serve] Add IP addresses to about page | Natalie Pendragon | 1 | +6 | -3 |
| 2020-08-05 13:03 | [threads] Add different sort orders for threads | Natalie Pendragon | 3 | +44 | -4 |
| 2020-08-03 16:55 | [serve] Improve feed matching | Natalie Pendragon | 1 | +4 | -0 |
| 2020-08-02 13:51 | Update naming | Natalie Pendragon | 6 | +5 | -13 |
| 2020-08-02 13:46 | [crawl] Improve handling of change_frequency | Natalie Pendragon | 3 | +87 | -23 |
| 2020-08-02 09:45 | [serve] Add Known Feeds page | Natalie Pendragon | 6 | +37 | -2 |
| 2020-08-02 09:42 | [threads] Add collapsible log variations | Natalie Pendragon | 5 | +55 | -11 |
| 2020-07-28 12:56 | [threads] Fix thread ordering | Natalie Pendragon | 1 | +4 | -4 |
| 2020-07-28 11:04 | [crawl] Index more errors | Natalie Pendragon | 1 | +8 | -2 |
| 2020-07-28 11:04 | [crawl] Add change_frequency backoff | Natalie Pendragon | 1 | +13 | -4 |
| 2020-07-28 11:03 | Bump dependencies | Natalie Pendragon | 1 | +4 | -4 |
| 2020-07-28 11:02 | Add friendly authors and titles for threads | Natalie Pendragon | 5 | +100 | -11 |
| 2020-07-27 18:50 | Threads v1 | Natalie Pendragon | 8 | +271 | -19 |
| 2020-07-24 10:43 | [serve] Save searches to db | Natalie Pendragon | 2 | +12 | -3 |
| 2020-07-23 18:40 | [build_index] [serve] Distinguish cross-capsule backlinks | Natalie Pendragon | 8 | +69 | -18 |
| 2020-07-23 13:44 | [crawl] Add is_cross_host_like field to db | Natalie Pendragon | 4 | +47 | -2 |
| 2020-07-23 12:35 | Gitignore all the indexes | Natalie Pendragon | 1 | +1 | -2 |
| 2020-07-23 12:29 | Bump dependencies | Natalie Pendragon | 1 | +47 | -46 |
| 2020-07-23 10:54 | Create scripts directory | Natalie Pendragon | 6 | +175 | -2 |
| 2020-07-22 17:29 | Add normalized url to db | Natalie Pendragon | 5 | +44 | -38 |
| 2020-07-21 19:43 | [serve] Add cert change to news page | Natalie Pendragon | 1 | +3 | -0 |
| 2020-07-21 18:49 | [build_index] Account for per-page expiration | Natalie Pendragon | 2 | +31 | -11 |
| 2020-07-20 12:19 | [build_index] Build index with backlink_count instead of backlinks | Natalie Pendragon | 3 | +22 | -17 |
| 2020-07-20 11:56 | [crawl] Start indexing errors | Natalie Pendragon | 4 | +60 | -4 |
| 2020-07-19 13:23 | [crawl] Update db model, and delete links before recreating | Natalie Pendragon | 2 | +4 | -3 |
| 2020-07-19 12:18 | [crawl] Ensure manual exclusions stay out of the database | Natalie Pendragon | 1 | +7 | -0 |
| 2020-07-19 11:35 | [serve] minor formatting updates | Natalie Pendragon | 2 | +2 | -2 |
| 2020-07-19 11:32 | [crawl] Support per-page expiration | Natalie Pendragon | 4 | +130 | -109 |
| 2020-07-15 13:09 | [crawl] Rebuild link table completely and idempotently | Natalie Pendragon | 2 | +12 | -2 |
| 2020-07-15 12:20 | [serve] Get backlinks from db instead of index | Natalie Pendragon | 1 | +12 | -11 |
| 2020-07-13 23:55 | [crawl] Set cap on maxiumum redirect chain length | Natalie Pendragon | 2 | +12 | -2 |
| 2020-07-13 23:18 | [crawl] Abort when detecting self-redirects | Natalie Pendragon | 1 | +5 | -1 |
| 2020-07-13 23:17 | [crawl] Ignore 80h gopher proxy | Natalie Pendragon | 1 | +3 | -0 |
| 2020-07-12 13:27 | [serve] Improve pager linking back to previous page | Natalie Pendragon | 1 | +3 | -1 |
| 2020-07-11 12:33 | [serve] Update backlinks links and presentation throughout GUS | Natalie Pendragon | 5 | +10 | -5 |
| 2020-07-11 10:56 | [serve] Improve safety of backlinks code path | Natalie Pendragon | 1 | +2 | -0 |
| 2020-07-08 10:18 | [crawl] Add feature to seed incremental crawl with atom feeds | Natalie Pendragon | 4 | +152 | -17 |
| 2020-07-06 10:22 | Make incremental build_index work | Natalie Pendragon | 2 | +23 | -11 |
| 2020-07-06 10:20 | DRY up the sqlite model and init_db code | Natalie Pendragon | 5 | +54 | -91 |
| 2020-07-05 12:52 | [serve] Improve handling of backlink searches | Natalie Pendragon | 1 | +11 | -2 |
| 2020-07-05 12:02 | [serve] Add historical statistics page | Natalie Pendragon | 5 | +36 | -13 |
| 2020-07-05 11:01 | [crawl] [serve] Run statistics and domains from sqlite db | Natalie Pendragon | 4 | +53 | -54 |
| 2020-07-04 10:43 | Improve discovery of backlinks | Natalie Pendragon | 2 | +13 | -6 |
| 2020-07-03 15:45 | [serve] Fix minor bug in counting of backlinks | Natalie Pendragon | 1 | +2 | -2 |
| 2020-07-03 14:39 | [crawl] [serve] Switch crawl to 2-phase with sqlite | Natalie Pendragon | 7 | +382 | -185 |
| 2020-06-30 12:57 | [crawl] Ignore localhost | Natalie Pendragon | 1 | +1 | -0 |
| 2020-06-30 12:54 | [serve] Add backlinks news and documentation | Natalie Pendragon | 2 | +11 | -0 |
| 2020-06-30 12:28 | [serve] Improve verbose mode | Natalie Pendragon | 3 | +20 | -13 |
| 2020-06-30 12:24 | [serve] Update header levels | Natalie Pendragon | 6 | +26 | -23 |
| 2020-06-30 11:07 | [crawl] [serve] Add backlinks | Natalie Pendragon | 6 | +94 | -11 |
| 2020-06-22 20:57 | [crawl] Ignore more bad content | Natalie Pendragon | 1 | +10 | -0 |
| 2020-06-18 11:16 | Update README | Natalie Pendragon | 1 | +3 | -20 |
| 2020-06-18 10:58 | [serve] Rearchitect serve to use templates and MVC pattern | Natalie Pendragon | 20 | +543 | -496 |
| 2020-06-17 13:09 | Add GUS licence | Natalie Pendragon | 1 | +33 | -0 |
| 2020-06-17 11:36 | [serve] Make seed request handling async again for now | Natalie Pendragon | 1 | +6 | -5 |
| 2020-06-17 11:33 | [crawl] Ignore some more alexschroeder pages | Natalie Pendragon | 1 | +27 | -1 |
| 2020-06-12 13:38 | [serve] Sort domains on the known-hosts page | Natalie Pendragon | 2 | +3 | -2 |
| 2020-06-12 10:40 | [serve] Add size to result rendering | Natalie Pendragon | 2 | +91 | -11 |
| 2020-06-11 10:38 | [crawl] Start indexing response sizes | Natalie Pendragon | 2 | +12 | -2 |
| 2020-06-10 12:09 | [serve] Use preformatted blocks on the statistics page | Natalie Pendragon | 1 | +8 | -2 |
| 2020-06-09 11:01 | Bump dependencies | Natalie Pendragon | 1 | +29 | -29 |
| 2020-06-09 10:55 | [crawl] Start indexing lang parameter | Natalie Pendragon | 2 | +16 | -10 |
| 2020-06-08 11:29 | [serve] Update some copy on about page | Natalie Pendragon | 1 | +1 | -1 |
| 2020-06-08 11:28 | Revert "[crawl] Index raw content for regex searches" | Natalie Pendragon | 1 | +0 | -2 |
| 2020-06-07 12:32 | [crawl] Ignore some more things | Natalie Pendragon | 1 | +21 | -0 |
| 2020-06-07 11:05 | [crawl] Add marmaladefoo's calculator to manual exclusions | Natalie Pendragon | 1 | +3 | -0 |
| 2020-06-05 11:35 | Add easy CLI way of removing domains from index | Natalie Pendragon | 2 | +41 | -0 |
| 2020-06-05 10:46 | [crawl] Remove manual exclusions for alexschroeder.ch | Natalie Pendragon | 1 | +0 | -10 |
| 2020-06-05 10:41 | [crawl] Add custom crawl delays | Natalie Pendragon | 1 | +7 | -2 |
| 2020-06-04 15:27 | [crawl] Improve indexing performance | Natalie Pendragon | 1 | +31 | -45 |
| 2020-06-03 23:37 | Update some seeds | Natalie Pendragon | 1 | +2 | -1 |
| 2020-06-03 20:28 | [crawl] Start indexing the charset | Natalie Pendragon | 4 | +56 | -8 |
| 2020-06-03 16:50 | [crawl] Only attempt to extract contained resources from text/gemini | Natalie Pendragon | 1 | +8 | -4 |
| 2020-06-03 16:50 | [crawl] Ignore some troublesome content from alexschroeder.ch | Natalie Pendragon | 1 | +10 | -0 |
| 2020-06-03 16:50 | [crawl] Fix default crawl delay when not specified explicitly | Natalie Pendragon | 1 | +4 | -4 |
| 2020-06-03 14:58 | [crawl] Persist index & crawl statistics on non-destructive crawls | Natalie Pendragon | 2 | +14 | -12 |
| 2020-06-03 14:53 | Bump dependency versions | Natalie Pendragon | 1 | +11 | -11 |
| 2020-06-03 14:49 | [crawl] Index raw content for regex searches | Natalie Pendragon | 1 | +3 | -1 |
| 2020-06-03 14:47 | [serve] Use "OR" as the default connector for queries | Natalie Pendragon | 1 | +13 | -3 |
| 2020-05-29 18:40 | [serve] Make sure two closely-timed seed requests don't break | Natalie Pendragon | 1 | +11 | -2 |
| 2020-05-28 13:02 | [crawl] Improve hierarchical handling of robots.txt entries | Natalie Pendragon | 1 | +12 | -4 |
| 2020-05-26 13:48 | [serve] Update copy on known hosts page | Natalie Pendragon | 1 | +1 | -1 |
| 2020-05-26 10:57 | [crawl] Ignore some Geddit URL prefixes | Natalie Pendragon | 1 | +4 | -0 |
| 2020-05-26 01:44 | [crawl] [serve] Add fetchable URL to the index | Natalie Pendragon | 2 | +7 | -2 |
| 2020-05-25 17:19 | Bump version of Jetforce dependency | Natalie Pendragon | 1 | +3 | -3 |
| 2020-05-25 10:31 | [crawl] Improve handling of quoting and unquoting URLs | Natalie Pendragon | 1 | +9 | -3 |
| 2020-05-25 03:05 | Rename fully_qualified_url to fetchable_url | Natalie Pendragon | 2 | +19 | -19 |
| 2020-05-25 03:00 | Rename fully_qualified_massaged_url to indexable_url | Natalie Pendragon | 2 | +11 | -11 |
| 2020-05-25 02:54 | [crawl] Fix bug in fully_qualified_massaged_url | Natalie Pendragon | 2 | +2 | -2 |
| 2020-05-24 14:08 | [crawl] Stop storing responses in GeminiResource objects | Natalie Pendragon | 2 | +33 | -37 |
| 2020-05-24 14:10 | Bump version of gusmobile dependency | Natalie Pendragon | 1 | +1 | -1 |
| 2020-05-24 11:28 | [crawl] Handle url fragments | Natalie Pendragon | 1 | +8 | -1 |
| 2020-05-23 13:11 | [crawl] Fix handling of robots.txt | Natalie Pendragon | 2 | +63 | -50 |
| 2020-05-23 11:19 | [crawl] Exclude "rss.xml" paths | Natalie Pendragon | 1 | +1 | -0 |
| 2020-05-22 13:18 | [crawl] Optimize the index after crawls | Natalie Pendragon | 1 | +3 | -0 |
| 2020-05-22 12:42 | [serve] Update highlight scoring and rendering | Natalie Pendragon | 2 | +9 | -4 |
| 2020-05-22 11:31 | [crawl] pickle and unpickle the robot_file_map | Natalie Pendragon | 2 | +14 | -3 |
| 2020-05-22 11:20 | Improve handling of unquoting URLs | Natalie Pendragon | 1 | +1 | -4 |
| 2020-05-21 20:07 | [serve] Update documentation on filters | Natalie Pendragon | 1 | +17 | -6 |
| 2020-05-21 19:35 | Update locked version of Gusmobile | Natalie Pendragon | 1 | +4 | -4 |
| 2020-05-21 14:59 | [crawl] Add domain field to index | Natalie Pendragon | 1 | +6 | -0 |
| 2020-05-21 13:25 | Remove outdated TODO | Natalie Pendragon | 1 | +0 | -3 |
| 2020-05-21 13:18 | [serve] Update formatting of statistics page | Natalie Pendragon | 1 | +2 | -3 |
| 2020-05-21 12:39 | [serve] Fix bug with first/next/previous page link formatting | Natalie Pendragon | 1 | +4 | -3 |
| 2020-05-21 11:57 | [serve] Only highlight nice content types in search results | Natalie Pendragon | 1 | +1 | -1 |
| 2020-05-21 11:33 | [crawl] Make path exclusions more robust | Natalie Pendragon | 1 | +4 | -4 |
| 2020-05-21 10:53 | [serve] Remove broken URL count from stats page | Natalie Pendragon | 1 | +0 | -1 |
| 2020-05-21 10:45 | Add houston to seeds, but ignore its search results | Natalie Pendragon | 1 | +5 | -0 |
| 2020-05-21 10:45 | [crawl] [serve] Add search highlights | Natalie Pendragon | 3 | +106 | -7 |
| 2020-05-20 13:33 | [crawl] Index massaged URLs | Natalie Pendragon | 2 | +27 | -13 |
| 2020-05-20 13:32 | [crawl] Handle trailing slash redirects better | Natalie Pendragon | 2 | +6 | -1 |
| 2020-05-20 12:15 | [serve] Update the loading of statistics | Natalie Pendragon | 1 | +21 | -6 |
| 2020-05-19 21:08 | [crawl] Fix lots of bugs | Natalie Pendragon | 4 | +124 | -103 |
| 2020-05-19 10:47 | [crawl] Crawl the seed requests after the main crawl | Natalie Pendragon | 1 | +15 | -0 |
| 2020-05-19 10:36 | [crawl] Fix bug in relative URL parsing | Natalie Pendragon | 1 | +2 | -2 |
| 2020-05-18 19:52 | [crawl] Fix bug with computing full_qualified_urls | Natalie Pendragon | 3 | +29 | -10 |
| 2020-05-18 13:12 | [crawl] Use standardized print_index_statistics | Natalie Pendragon | 2 | +13 | -19 |
| 2020-05-18 13:01 | [no-op] Clean up comments in whoosh_extensions | Natalie Pendragon | 1 | +0 | -3 |
| 2020-05-18 12:57 | [serve] Crawl and index seed requests immediately | Natalie Pendragon | 4 | +65 | -18 |
| 2020-05-17 14:30 | Update README TODOs | Natalie Pendragon | 1 | +6 | -8 |
| 2020-05-17 14:20 | [crawl] Implement GeminiResource | Natalie Pendragon | 5 | +167 | -86 |
| 2020-05-17 11:45 | [crawl] Exclude GUS search result pages from crawl | Natalie Pendragon | 1 | +2 | -0 |
| 2020-05-17 10:21 | [crawl] Add seeds | Natalie Pendragon | 1 | +3 | -0 |
| 2020-05-16 18:51 | [crawl] Add jan.bio to seeds | Natalie Pendragon | 1 | +1 | -0 |
| 2020-05-16 15:23 | Add index.bak to gitignore | Natalie Pendragon | 1 | +1 | -0 |
| 2020-05-16 14:57 | [crawl] Create non-destructive crawl option | Natalie Pendragon | 3 | +33 | -7 |
| 2020-05-16 13:23 | [serve] Improve documentation on content type queries | Natalie Pendragon | 1 | +5 | -13 |
| 2020-05-16 13:05 | [serve] Add verbose mode | Natalie Pendragon | 2 | +51 | -13 |
| 2020-05-16 12:22 | [serve] Update how num_results is displayed | Natalie Pendragon | 1 | +4 | -4 |
| 2020-05-16 12:12 | [serve] Improve search result data type | Natalie Pendragon | 1 | +13 | -5 |
| 2020-05-16 12:00 | [crawl] [serve] Add more statistics | Natalie Pendragon | 4 | +55 | -20 |
| 2020-05-16 10:57 | [crawl] Update seeds | Natalie Pendragon | 1 | +4 | -0 |
| 2020-05-15 12:03 | [crawl] Update seeds | Natalie Pendragon | 1 | +5 | -1 |
| 2020-05-15 12:01 | Update and reorder TODOs | Natalie Pendragon | 1 | +14 | -9 |
| 2020-05-15 10:27 | [crawl] [no-op] Add a line after backup operation | Natalie Pendragon | 1 | +1 | -0 |
| 2020-05-14 19:40 | Update statistics TODOs | Natalie Pendragon | 1 | +5 | -1 |
| 2020-05-14 13:17 | [crawl] Add new seed | Natalie Pendragon | 1 | +1 | -0 |
| 2020-05-14 12:49 | [serve] Update statistics copy slightly | Natalie Pendragon | 1 | +2 | -2 |
| 2020-05-14 11:56 | [serve] Implement paging | Natalie Pendragon | 2 | +29 | -16 |
| 2020-05-14 10:59 | Update README ideas for more index/usage statistics | Natalie Pendragon | 1 | +7 | -4 |
| 2020-05-13 14:20 | [crawl] Add new spanish site to crawl seeds | Natalie Pendragon | 1 | +4 | -0 |
| 2020-05-13 13:51 | [crawl] Refactor manual exclusions and add fgaz' calculator | Natalie Pendragon | 1 | +13 | -4 |
| 2020-05-12 12:52 | Add TODO for generating and sharing GUS usage statistics | Natalie Pendragon | 1 | +5 | -0 |
| 2020-05-12 12:46 | [serve] Add news feature | Natalie Pendragon | 1 | +43 | -1 |
| 2020-05-12 12:18 | [serve] Add page to show all known hosts | Natalie Pendragon | 1 | +22 | -0 |
| 2020-05-12 11:56 | [statistics] Add ability to compute and print stats easily | Natalie Pendragon | 2 | +29 | -5 |
| 2020-05-12 11:23 | [statistics] Refactor statistics objects to pass around dicts | Natalie Pendragon | 2 | +25 | -25 |
| 2020-05-12 11:07 | [serve] Add page headers | Natalie Pendragon | 1 | +8 | -1 |
| 2020-05-11 18:51 | [serve] Update copy for current index statistics | Natalie Pendragon | 1 | +2 | -2 |
| 2020-05-11 18:45 | [serve] Stop hard-wrapping content | Natalie Pendragon | 1 | +7 | -18 |
| 2020-05-11 17:56 | [serve] Report out current index statistics | Natalie Pendragon | 4 | +56 | -5 |
| 2020-05-11 17:16 | Refactor some common/library code into separate files | Natalie Pendragon | 7 | +86 | -70 |
| 2020-05-10 16:12 | [serve] Remove TODO to add documentation for content_type | Natalie Pendragon | 1 | +0 | -1 |
| 2020-05-10 15:50 | [crawl] Alphabetize and add a few more seeds | Natalie Pendragon | 1 | +25 | -16 |
| 2020-05-10 14:39 | [crawl] Backup old index before running crawl | Natalie Pendragon | 1 | +9 | -0 |
| 2020-05-10 14:38 | [crawl] Add indexed_at field | Natalie Pendragon | 2 | +7 | -2 |
| 2020-05-09 21:34 | [crawl] Compute and generate index statistics after each crawl | Natalie Pendragon | 1 | +57 | -1 |
| 2020-05-09 21:23 | [serve] Update content_type search documentation | Natalie Pendragon | 1 | +5 | -1 |
| 2020-05-09 20:05 | Add TODO to track Geminispace statistics | Natalie Pendragon | 1 | +5 | -0 |
| 2020-05-09 18:07 | [serve] Add documentation for content_types | Natalie Pendragon | 1 | +24 | -4 |
| 2020-05-09 17:35 | [serve] Add note that paging isn't implemented yet | Natalie Pendragon | 1 | +1 | -1 |
| 2020-05-09 17:35 | [serve] Put index generation date in footer | Natalie Pendragon | 2 | +11 | -2 |
| 2020-05-09 16:38 | Add a couple TODOs | Natalie Pendragon | 1 | +2 | -0 |
| 2020-05-09 15:54 | [crawl] Add two new seeds | Natalie Pendragon | 1 | +2 | -0 |
| 2020-05-09 15:06 | [crawl] Stop printing the sleep duration | Natalie Pendragon | 1 | +0 | -1 |
| 2020-05-09 15:00 | [crawl] Improve error recovery | Natalie Pendragon | 1 | +35 | -24 |
| 2020-05-09 14:58 | [crawl] Adjust link line regex to only match at beginning of line | Natalie Pendragon | 2 | +6 | -2 |
| 2020-05-05 12:27 | [crawl] Respect robots.txt crawl_delays and add a kind default | Natalie Pendragon | 2 | +29 | -8 |
| 2020-04-17 13:24 | Add some TODOs | Natalie Pendragon | 1 | +4 | -0 |
| 2020-04-16 22:40 | [serve] Fix bug in displaying "input" results | Natalie Pendragon | 1 | +2 | -2 |
| 2020-04-16 22:39 | Update dependencies | Natalie Pendragon | 1 | +49 | -49 |
| 2020-04-16 22:19 | [crawl] fix crawl bug with robots.txt | Natalie Pendragon | 1 | +2 | -2 |
| 2020-04-16 22:18 | [serve] Update formatting | Natalie Pendragon | 1 | +3 | -7 |
| 2020-03-15 02:50 | Improve it all | Natalie Pendragon | 4 | +106 | -81 |
| 2020-03-05 13:55 | [serve] Add seed request tracking | Natalie Pendragon | 2 | +21 | -0 |
| 2020-03-05 12:50 | [serve] Update aesthetics | Natalie Pendragon | 1 | +12 | -12 |
| 2020-03-04 13:08 | Add search suggestions | Natalie Pendragon | 1 | +36 | -5 |
| 2020-03-04 13:08 | Update indexing and query parsing | Natalie Pendragon | 3 | +36 | -6 |
| 2020-03-04 13:06 | Add TODO to track freshness of content | Natalie Pendragon | 1 | +1 | -0 |
| 2020-03-02 11:43 | [crawl] Respect "indexer" robots.txt entries | Natalie Pendragon | 1 | +1 | -1 |
| 2020-03-01 17:12 | Add more feature ideas to the README | Natalie Pendragon | 1 | +9 | -0 |
| 2020-03-01 17:12 | Index and serve mime types | Natalie Pendragon | 2 | +4 | -2 |
| 2020-02-29 13:33 | Improve README readability | Natalie Pendragon | 1 | +6 | -6 |
| 2020-02-29 13:31 | Add README todo to add paging | Natalie Pendragon | 1 | +7 | -0 |
| 2020-02-29 13:27 | [serve] Remove numbers from search result rows | Natalie Pendragon | 1 | +1 | -1 |
| 2020-02-29 13:13 | Update README.md | Natalie Pendragon | 1 | +27 | -1 |
| 2020-02-27 14:06 | Update README | Natalie Pendragon | 2 | +24 | -19 |
| 2020-02-27 13:45 | Make GUS easier to run for others | Natalie Pendragon | 3 | +60 | -43 |
| 2020-02-23 14:30 | Add some new seed sites | Natalie Pendragon | 1 | +3 | -1 |
| 2020-02-21 13:44 | Respect robots.txt | Natalie Pendragon | 2 | +41 | -10 |
| 2020-01-30 13:47 | Initial commit | Natalie Pendragon | 6 | +1024 | -0 |
