geminispace.info

gemini search engine
git clone https://git.clttr.info/geminispace.info.git
Log (Feed) | Files | Refs (Tags) | README | LICENSE

commit 739328b9e5aae6855f1cc92def9872dd5ba50741
parent 5a28dae35d06dbf00398e05584e6cf612951cfdb
Author: Rene Wagner <rwa@clttr.info>
Date:   Wed,  8 Mar 2023 19:02:57 +0000

update README

Diffstat:
MREADME.md | 43++++++++++++++++++++++++++++++-------------
Minfra/gus.service | 12+++++-------
Minfra/rebuild_index.sh | 10+++++-----
Minfra/update_index.sh | 8++++----
4 files changed, 44 insertions(+), 29 deletions(-)

diff --git a/README.md b/README.md @@ -3,8 +3,17 @@ ## Dependencies -1. Install python (>3.5) and [poetry](https://python-poetry.org) -2. Run: `poetry install` +Install the following packages +- Python (>3.5) including `distutils` +- [poetry](https://python-poetry.org) +- OpenSSL +- SQLite3 + +## basic setup + +1. Create a user who should run the service, e.g. `gus` +2. Clone this repo to a directory of your choice +3. Run `poetry install` in this directory ## Making an initial index @@ -12,33 +21,41 @@ Make sure you have some gemini URLs for testing which are nicely sandboxed to avoid indexing huge parts of the gemini space. -1. Create a "seed-requests.txt" file with you test gemini URLs -2. Run: `poetry run crawl -d` -3. Run: `poetry run build_index -d` +1. Create a "seed-requests.txt" file with your test gemini URLs +2. Run `poetry run crawl -d` (might take a few hours) +3. Run `poetry run build_index -d` Now you'll have created `index.new` directory, rename it to `index`. ## Running the frontend -1. Run: `poetry run serve` +1. Run `poetry run serve` 2. Navigate your gemini client to: "gemini://localhost/" ### Running the frontend in production with systemd -1. update `infra/gus.service` to match your needs (directory, user) -2. copy `infra/gus.service` to `/etc/systemd/system/` -3. run `systemctl enable gus` and `systemctl start gus` +1. Create a private key and TLS certificate (X.509 v3) for your host +2. Update `infra/gus.service` to match your needs: + - working directory + - domain + - certificate files +3. Copy `infra/gus.service` to `~/.config/systemd/user/` +4. Run the following commands to automatically start the daemon + - `systemctl --user daemon-reload` + - `systemctl --user enable gus` + - `systemctl --user start gus` +5. As root, run `loginctl enable-linger gus` ## Running the crawl to update the index -1. Run: `poetry run crawl` -2. Run: `poetry run build_index` -3. Restart frontend +1. Run `poetry run crawl` +2. Run `poetry run build_index` +3. Run `systemctl --user restart gus` ### Running the crawl & indexer in production with systemd -3. set up a cron job with the following params: `0 9 * * * /home/gus/infra/update-index.sh /home/gus` +3. set up a cron job with the following params: `0 9 * * * <path to your working dir>/infra/update-index.sh <path to your working dir>` ## Running the test suite diff --git a/infra/gus.service b/infra/gus.service @@ -1,21 +1,19 @@ -# /etc/systemd/system/gus.service +# /home/gus/.config/systemd/gus.service [Unit] Description=Gemini Universal Search [Service] -User=gus -Group=gus Type=simple Restart=always RestartSec=5 -WorkingDirectory=/home/gus/ +WorkingDirectory=/home/gus/geminispace Environment="PYTHONUNBUFFERED=1" -ExecStart=/home/gus/.poetry/bin/poetry run serve \ +ExecStart=/home/gus/.local/bin/poetry run serve \ --port 1965 \ --host "::" \ - --hostname geminispace.info \ - --tls-certfile geminispace.info.crt \ + --hostname test.geminispace.info \ + --tls-certfile geminispace.info.v3.crt \ --tls-keyfile geminispace.info.key [Install] diff --git a/infra/rebuild_index.sh b/infra/rebuild_index.sh @@ -1,17 +1,17 @@ #!/bin/bash -BASEDIR=${1} +BASEDIR=$1 LOCKFILE=${BASEDIR}/crawl.lock touch ${LOCKFILE} mkdir -p ${BASEDIR}/index.new/ -sudo systemctl stop gus +systemctl --user stop gus cp ${BASEDIR}/index/gus.sqlite ${BASEDIR}/index.new/ -sudo systemctl start gus -${BASEDIR}/.poetry/bin/poetry run build_index -d +systemctl --user start gus +~/.poetry/bin/poetry run build_index -d mv ${BASEDIR}/index ${BASEDIR}/index.old mv ${BASEDIR}/index.new ${BASEDIR}/index -sudo systemctl restart gus +systemctl --user restart gus rm ${LOCKFILE} diff --git a/infra/update_index.sh b/infra/update_index.sh @@ -1,5 +1,5 @@ #!/bin/bash -BASEDIR=${1} +BASEDIR=$1 LOCKFILE=${BASEDIR}/crawl.lock @@ -13,8 +13,8 @@ touch ${LOCKFILE} rm ${BASEDIR}/gus.log ${BASEDIR}/infra/filter_seed_request.sh ${BASEDIR} -${BASEDIR}/.poetry/bin/poetry run crawl -${BASEDIR}/.poetry/bin/poetry run build_index -sudo systemctl restart gus +~/.local/bin/poetry run crawl +~/.local/bin/poetry run build_index +systemctl --user restart gus rm ${LOCKFILE}