commit 382b6facc678e2df1e4fe8f356a4af691fdeb7d2 parent 892b841f86d93848d5d9cfa7c497706fca6d1c84 Author: Natalie Pendragon <natpen@natpen.net> Date: Fri, 17 Apr 2020 09:24:57 -0400 Add some TODOs Diffstat:
| M | README.md | | | 4 | ++++ |
1 file changed, 4 insertions(+), 0 deletions(-)
diff --git a/README.md b/README.md @@ -27,6 +27,10 @@ as this guide to [mailing list etiquette](https://man.sr.ht/lists.sr.ht/etiquett # Roadmap / TODOs +- **log output of crawl**: I see some errors fly by, and it + would be nice to be able to review later and investigate. +- **add support for delays in crawl**: be nicer to domain + owners and don't hit their sites too fast. - **general code cleanup**: most notably crawl.py. There are a lot of hacks in there that I put in for expediency, but haven't taken the time to address.