geminispace.info

commit 09f4969216e259eb57323cdb1f4d5031529acbec
parent 5fbaa742a0054bd3551ee86e3666c29011ce56a8
Author: René Wagner <rwa@clttr.info>
Date:   Sat, 29 Jul 2023 10:12:25 +0200

update news

Diffstat:
M
serve/templates/about.gmi
 | 
2
+-
M
serve/templates/documentation/indexing.gmi
 | 
2
++
M
serve/templates/news.gmi
 | 
4
++++

3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/serve/templates/about.gmi b/serve/templates/about.gmi
@@ -16,7 +16,7 @@ If you'd like to help keep geminispace.info running feel free to donate a few bu
 => https://liberapay.com/rwa/ support me on LiberaPay
 => https://donate.stripe.com/fZe3cr3386VnfzaaEF one time donation via Stripe
 
-In 2023 geminispace.info received donations worth 0f 105.92€ (transaction fees already excluded).
+In 2023 geminispace.info received donations worth of 105.92€ (transaction fees already excluded).
 In 2022 geminispace.info received donations worth of 82.78€.
 
 ### index updates
diff --git a/serve/templates/documentation/indexing.gmi b/serve/templates/documentation/indexing.gmi
@@ -8,6 +8,8 @@
 
 geminispace.info is a search engine for all content served over the Gemini Protocol. It can help you track down textual pages (e.g. `text/gemini`, `text/plain`, `text/markdown`) with content containing your search terms, but it can just as easily help you track down binary files (e.g., images, mp3s) which happen to be served over the Gemini protocol.
 
+The main purpose of geminispace.info is to index "native" Gemini content. We are not interested in indexing mirrors of news sites, wikipedia and such. We will most likely exclude thos capsules if no proper robots.txt is in place.
+
 ### What does geminispace.info index?
 
 geminispace.info will only index content within Geminispace, and will neither follow nor index links out to other protocols, like Http or Gopher. We will only crawl outwards by following Gemini links found within `text/gemini` pages. If you return a `text/plain` mimetype for a page, Gemini links within it will not register with GUS (though the content of the `text/plain` page will itself get indexed).
diff --git a/serve/templates/news.gmi b/serve/templates/news.gmi
@@ -2,6 +2,10 @@
 
 
 ## News
+### 2023-07-29 robots.txt
+Dear capsule pilots (especially those who run mirrors or gateways of bloated web site): Please put a robots.txt in place.
+=> gemini://gemini.circumlunar.space/docs/companion/robots.gmi robots.txt companion spec at circumlunar
+
 ### 2023-06-07 server switch
 Be welcome on our brand new production instance sporting Debian 12.