Reorganize codebase

This commit is contained in:
Izalia Mae 2022-05-06 07:04:51 +00:00
parent 5c5f212d70
commit b6494849b5
28 changed files with 1938 additions and 988 deletions

View file

@ -1,11 +1,28 @@
FROM python:3-alpine
WORKDIR /workdir
# install build deps for pycryptodome and other c-based python modules
RUN apk add alpine-sdk autoconf automake libtool gcc
ADD requirements.txt /workdir/
RUN pip3 install -r requirements.txt
# add env var to let the relay know it's in a container
ENV DOCKER_RUNNING=true
ADD . /workdir/
# setup various container properties
VOLUME ["/data"]
CMD ["python", "-m", "relay"]
EXPOSE 8080/tcp
WORKDIR /opt/activityrelay
VOLUME ["/workdir/data"]
# install and update important python modules
RUN pip3 install -U setuptools wheel pip
# only copy necessary files
COPY relay ./relay
COPY LICENSE .
COPY README.md .
COPY requirements.txt .
COPY setup.cfg .
COPY setup.py .
COPY .git ./.git
# install relay deps
RUN pip3 install -r requirements.txt

View file

@ -10,72 +10,14 @@ Affero General Public License version 3 (AGPLv3) license. You can find a copy o
in this package as the `LICENSE` file.
## Setup
You need at least Python 3.6 (latest version of 3.x recommended) to make use of this software.
It simply will not run on older Python versions.
Download the project and install with pip (`pip3 install .`).
Copy `relay.yaml.example` to `relay.yaml` and edit it as appropriate:
$ cp relay.yaml.example relay.yaml
$ $EDITOR relay.yaml
Finally, you can launch the relay:
$ python3 -m relay
It is suggested to run this under some sort of supervisor, such as runit, daemontools,
s6 or systemd. Configuration of the supervisor is not covered here, as it is different
depending on which system you have available.
The bot runs a webserver, internally, on localhost at port 8080. This needs to be
forwarded by nginx or similar. The webserver is used to receive ActivityPub messages,
and needs to be secured with an SSL certificate inside nginx or similar. Configuration
of your webserver is not discussed here, but any guide explaining how to configure a
modern non-PHP web application should cover it.
## Getting Started
Normally, you would direct your LitePub instance software to follow the LitePub actor
found on the relay. In Pleroma this would be something like:
$ MIX_ENV=prod mix relay_follow https://your.relay.hostname/actor
Mastodon uses an entirely different relay protocol but supports LitePub relay protocol
as well when the Mastodon relay handshake is used. In these cases, Mastodon relay
clients should follow `http://your.relay.hostname/inbox` as they would with Mastodon's
own relay software.
$ MIX_ENV=prod mix relay_follow https://your.relay.hostname/actor
## Performance
## Documentation
Performance is very good, with all data being stored in memory and serialized to a
JSON-LD object graph. Worker coroutines are spawned in the background to distribute
the messages in a scatter-gather pattern. Performance is comparable to, if not
superior to, the Mastodon relay software, with improved memory efficiency.
## Management
You can perform a few management tasks such as peering or depeering other relays by
invoking the `relay.manage` module.
This will show the available management tasks:
$ python3 -m relay.manage
When following remote relays, you should use the `/actor` endpoint as you would in
Pleroma and other LitePub-compliant software.
## Docker
You can run ActivityRelay with docker. Edit `relay.yaml` so that the database
location is set to `./data/relay.jsonld` and then build and run the docker
image :
$ docker volume create activityrelay-data
$ docker build -t activityrelay .
$ docker run -d -p 8080:8080 -v activityrelay-data:/workdir/data activityrelay
To install or manage your relay, check the [documentation](docs/index.md)

66
docker.sh Executable file
View file

@ -0,0 +1,66 @@
#!/usr/bin/env bash
case $1 in
install)
docker build -f Dockerfile -t activityrelay . && \
docker volume create activityrelay-data && \
docker run -it -p 8080:8080 -v activityrelay-data:/data --name activityrelay activityrelay
;;
uninstall)
docker stop activityrelay && \
docker container rm activityrelay && \
docker volume rm activityrelay-data && \
docker image rm activityrelay
;;
start)
docker start activityrelay
;;
stop)
docker stop activityrelay
;;
manage)
shift
docker exec -it activityrelay python3 -m relay "$@"
;;
shell)
docker exec -it activityrelay bash
;;
rescue)
docker run -it --rm --entrypoint bash -v activityrelay-data:/data activityrelay
;;
edit)
if [ -z ${EDITOR} ]; then
echo "EDITOR environmental variable not set"
exit
fi
CONFIG="/tmp/relay-$(date +"%T").yaml"
docker cp activityrelay:/data/relay.yaml $CONFIG && \
$EDITOR $CONFIG && \
docker cp $CONFIG activityrelay:/data/relay.yaml && \
rm $CONFIG
;;
*)
COLS="%-22s %s\n"
echo "Valid commands:"
printf "$COLS" "- start" "Run the relay in the background"
printf "$COLS" "- stop" "Stop the relay"
printf "$COLS" "- manage <cmd> [args]" "Run a relay management command"
printf "$COLS" "- edit" "Edit the relay's config in \$EDITOR"
printf "$COLS" "- shell" "Drop into a bash shell on the running container"
printf "$COLS" "- rescue" "Drop into a bash shell on a temp container with the data volume mounted"
printf "$COLS" "- install" "Build the image, create a new container and volume, and run relay setup"
printf "$COLS" "- uninstall" "Delete the relay image, container, and volume"
;;
esac

157
docs/commands.md Normal file
View file

@ -0,0 +1,157 @@
# Commands
There are a number of commands to manage your relay's database and config. You can add `--help` to
any category or command to get help on that specific option (ex. `activityrelay inbox --help`).
Note: Unless specified, it is recommended to run any commands while the relay is shutdown.
## Run
Run the relay.
activityrelay run
## Setup
Run the setup wizard to configure your relay.
activityrelay setup
## Inbox
Manage the list of subscribed instances.
### List
List the currently subscribed instances or relays.
activityrelay inbox list
### Add
Add an inbox to the database. If a domain is specified, it will default to `https://{domain}/inbox`.
If the added instance is not following the relay, expect errors when pushing messages.
activityrelay inbox add <inbox or domain>
### Remove
Remove an inbox from the database. An inbox or domain can be specified.
activityrelay inbox remove <inbox or domain>
### Follow
Follow an instance or relay actor and add it to the database. If a domain is specified, it will
default to `https://{domain}/actor`.
activityrelay inbox follow <actor or domain>
Note: The relay must be running for this command to work.
### Unfollow
Unfollow an instance or relay actor and remove it from the database. If the instance or relay does
not exist anymore, use the `inbox remove` command instead.
activityrelay inbox unfollow <domain, actor, or inbox>
Note: The relay must be running for this command to work.
## Whitelist
Manage the whitelisted domains.
### List
List the current whitelist.
activityrelay whitelist list
### Add
Add a domain to the whitelist.
activityrelay whitelist add <domain>
### Remove
Remove a domain from the whitelist.
activityrelay whitelist remove <domain>
## Instance
Manage the instance ban list.
### List
List the currently banned instances
activityrelay instance list
### Ban
Add an instance to the ban list. If the instance is currently subscribed, remove it from the
database.
activityrelay instance ban <domain>
### Unban
Remove an instance from the ban list.
activityrelay instance unban <domain>
## Software
Manage the software ban list. To get the correct name, check the software's nodeinfo endpoint.
You can find it at nodeinfo\['software']\['name'].
### List
List the currently banned software.
activityrelay software list
### Ban
Add a software name to the ban list.
If `-f` or `--fetch-nodeinfo` is set, treat the name as a domain and try to fetch the software
name via nodeinfo.
If the name is `RELAYS` (case-sensitive), add all known relay software names to the list.
activityrelay software ban [-f/--fetch-nodeinfo] <name, domain, or RELAYS>
### Unban
Remove a software name from the ban list.
If `-f` or `--fetch-nodeinfo` is set, treat the name as a domain and try to fetch the software
name via nodeinfo.
If the name is `RELAYS` (case-sensitive), remove all known relay software names from the list.
activityrelay unban [-f/--fetch-nodeinfo] <name, domain, or RELAYS>

110
docs/configuration.md Normal file
View file

@ -0,0 +1,110 @@
# Configuration
## DB
The path to the database. It contains the relay actor private key and all subscribed
instances. If the path is not absolute, it is relative to the working directory.
db: relay.jsonld
## Listener
The address and port the relay will listen on. If the reverse proxy (nginx, apache, caddy, etc)
is running on the same host, it is recommended to change `listen` to `localhost`
listen: 0.0.0.0
port: 8080
## Note
A small blurb to describe your relay instance. This will show up on the relay's home page.
note: "Make a note about your instance here."
## Post Limit
The maximum number of messages to send out at once. For each incoming message, a message will be
sent out to every subscribed instance minus the instance which sent the message. This limit
is to prevent too many outgoing connections from being made, so adjust if necessary.
push_limit: 512
## AP
Various ActivityPub-related settings
### Host
The domain your relay will use to identify itself.
host: relay.example.com
### Whitelist Enabled
If set to `true`, only instances in the whitelist can follow the relay. Any subscribed instances
not in the whitelist will be removed from the inbox list on startup.
whitelist_enabled: false
### Whitelist
A list of domains of instances which are allowed to subscribe to your relay.
whitelist:
- bad-instance.example.com
- another-bad-instance.example.com
### Blocked Instances
A list of instances which are unable to follow the instance. If a subscribed instance is added to
the block list, it will be removed from the inbox list on startup.
blocked_instances:
- bad-instance.example.com
- another-bad-instance.example.com
### Blocked Software
A list of ActivityPub software which cannot follow your relay. This list is empty by default, but
setting this to the above list will block all other relays and prevent relay chains
blocked_software:
- activityrelay
- aoderelay
- social.seattle.wa.us-relay
- unciarelay
## Cache
These are object limits for various caches. Only change if you know what you're doing.
### Objects
The urls of messages which have been processed by the relay.
objects: 1024
### Actors
The ActivityPub actors of incoming messages.
actors: 1024
### Actors
The base64 encoded hashes of messages.
digests: 1024

9
docs/index.md Normal file
View file

@ -0,0 +1,9 @@
# ActivityRelay Documentation
ActivityRelay is a small ActivityPub server that relays messages to subscribed instances.
[Installation](installation.md)
[Configuration](configuration.md)
[Commands](commands.md)

67
docs/installation.md Normal file
View file

@ -0,0 +1,67 @@
# Installation
There are a few ways to install ActivityRelay. Follow one of the methods below, setup a reverse
proxy, and setup the relay to run via a supervisor. Example configs for caddy, nginx, and systemd
in `installation/`
## Pipx
Pipx uses pip and a custom venv implementation to automatically install modules into a Python
environment and is the recommended method. Install pipx if it isn't installed already. Check out
the [official pipx docs](https://pypa.github.io/pipx/installation/) for more in-depth instructions.
python3 -m pip install pipx
Now simply install ActivityRelay directly from git
pipx install git+https://git.pleroma.social/pleroma/relay@0.2.0
Or from a cloned git repo.
pipx install .
Once finished, you can set up the relay via the setup command. It will ask a few questions to fill
out config options for your relay
activityrelay setup
Finally start it up with the run command.
activityrelay run
Note: Pipx requires python 3.7+. If your distro doesn't have a compatible version of python, it can
be installed via
## Pip
The instructions for installation via pip are very similar to pipx. Installation can be done from
git
python3 -m pip install git+https://git.pleroma.social/pleroma/relay@0.2.0
or a cloned git repo.
python3 -m pip install .
Now run the configuration wizard
activityrelay setup
And start the relay when finished
activityrelay run
## Docker
Installation and management via Docker can be handled with the `docker.sh` script. To install
ActivityRelay, run the install command. Once the image is built and the container is created,
your will be asked to fill out some config options for your relay.
./docker.sh install
Finally start it up. It will be listening on TCP port 8080.
./docker.sh start

View file

@ -9,18 +9,25 @@ port: 8080
# Note
note: "Make a note about your instance here."
# maximum number of inbox posts to do at once
post_limit: 512
# this section is for ActivityPub
ap:
# this is used for generating activitypub messages, as well as instructions for
# linking AP identities. it should be an SSL-enabled domain reachable by https.
host: 'relay.example.com'
blocked_instances:
- 'bad-instance.example.com'
- 'another-bad-instance.example.com'
whitelist_enabled: false
whitelist:
- 'good-instance.example.com'
- 'another.good-instance.example.com'
# uncomment the lines below to prevent certain activitypub software from posting
# to the relay (all known relays by default). this uses the software name in nodeinfo
#blocked_software:
@ -28,3 +35,9 @@ ap:
#- 'aoderelay'
#- 'social.seattle.wa.us-relay'
#- 'unciarelay'
# cache limits as number of items. only change this if you know what you're doing
cache:
objects: 1024
actors: 1024
digests: 1024

View file

@ -1,58 +1,8 @@
from . import logging
__version__ = '0.2.0'
from aiohttp.web import Application
from . import logger
import asyncio
import aiohttp
import aiohttp.web
import yaml
import argparse
parser = argparse.ArgumentParser(
description="A generic LitePub relay (works with all LitePub consumers and Mastodon).",
prog="python -m relay")
parser.add_argument("-c", "--config", type=str, default="relay.yaml",
metavar="<path>", help="the path to your config file")
args = parser.parse_args()
def load_config():
with open(args.config) as f:
options = {}
## Prevent a warning message for pyyaml 5.1+
if getattr(yaml, 'FullLoader', None):
options['Loader'] = yaml.FullLoader
yaml_file = yaml.load(f, **options)
config = {
'db': yaml_file.get('db', 'relay.jsonld'),
'listen': yaml_file.get('listen', '0.0.0.0'),
'port': int(yaml_file.get('port', 8080)),
'note': yaml_file.get('note', 'Make a note about your instance here.'),
'ap': {
'blocked_software': [v.lower() for v in yaml_file['ap'].get('blocked_software', [])],
'blocked_instances': yaml_file['ap'].get('blocked_instances', []),
'host': yaml_file['ap'].get('host', 'localhost'),
'whitelist': yaml_file['ap'].get('whitelist', []),
'whitelist_enabled': yaml_file['ap'].get('whitelist_enabled', False)
}
}
return config
CONFIG = load_config()
from .http_signatures import http_signatures_middleware
app = aiohttp.web.Application(middlewares=[
http_signatures_middleware
])
from . import database
from . import actor
from . import webfinger
from . import default
from . import nodeinfo
from . import http_stats
app = Application()

View file

@ -1,56 +1,5 @@
import asyncio
import aiohttp.web
import logging
import platform
import sys
import Crypto
import time
from . import app, CONFIG
def crypto_check():
vers_split = platform.python_version().split('.')
pip_command = 'pip3 uninstall pycrypto && pip3 install pycryptodome'
if Crypto.__version__ != '2.6.1':
return
if int(vers_split[1]) > 7 and Crypto.__version__ == '2.6.1':
logging.error('PyCrypto is broken on Python 3.8+. Please replace it with pycryptodome before running again. Exiting in 10 sec...')
logging.error(pip_command)
time.sleep(10)
sys.exit()
else:
logging.warning('PyCrypto is old and should be replaced with pycryptodome')
logging.warning(pip_command)
async def start_webserver():
runner = aiohttp.web.AppRunner(app)
await runner.setup()
try:
listen = CONFIG['listen']
except:
listen = 'localhost'
try:
port = CONFIG['port']
except:
port = 8080
logging.info('Starting webserver at {listen}:{port}'.format(listen=listen,port=port))
site = aiohttp.web.TCPSite(runner, listen, port)
await site.start()
def main():
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
asyncio.ensure_future(start_webserver(), loop=loop)
loop.run_forever()
from .manage import main
if __name__ == '__main__':
crypto_check()
main()
main()

View file

@ -1,347 +0,0 @@
import aiohttp
import aiohttp.web
import asyncio
import logging
import uuid
import re
import simplejson as json
import cgi
import datetime
from urllib.parse import urlsplit
from Crypto.PublicKey import RSA
from cachetools import LFUCache
from . import app, CONFIG
from .database import DATABASE
from .http_debug import http_debug
from .remote_actor import fetch_actor
from .http_signatures import sign_headers, generate_body_digest
# generate actor keys if not present
if "actorKeys" not in DATABASE:
logging.info("No actor keys present, generating 4096-bit RSA keypair.")
privkey = RSA.generate(4096)
pubkey = privkey.publickey()
DATABASE["actorKeys"] = {
"publicKey": pubkey.exportKey('PEM').decode('utf-8'),
"privateKey": privkey.exportKey('PEM').decode('utf-8')
}
PRIVKEY = RSA.importKey(DATABASE["actorKeys"]["privateKey"])
PUBKEY = PRIVKEY.publickey()
AP_CONFIG = CONFIG['ap']
CACHE_SIZE = CONFIG.get('cache-size', 16384)
CACHE = LFUCache(CACHE_SIZE)
sem = asyncio.Semaphore(500)
async def actor(request):
data = {
"@context": "https://www.w3.org/ns/activitystreams",
"endpoints": {
"sharedInbox": "https://{}/inbox".format(request.host)
},
"followers": "https://{}/followers".format(request.host),
"following": "https://{}/following".format(request.host),
"inbox": "https://{}/inbox".format(request.host),
"name": "ActivityRelay",
"type": "Application",
"id": "https://{}/actor".format(request.host),
"publicKey": {
"id": "https://{}/actor#main-key".format(request.host),
"owner": "https://{}/actor".format(request.host),
"publicKeyPem": DATABASE["actorKeys"]["publicKey"]
},
"summary": "ActivityRelay bot",
"preferredUsername": "relay",
"url": "https://{}/actor".format(request.host)
}
return aiohttp.web.json_response(data, content_type='application/activity+json')
app.router.add_get('/actor', actor)
get_actor_inbox = lambda actor: actor.get('endpoints', {}).get('sharedInbox', actor['inbox'])
async def push_message_to_actor(actor, message, our_key_id):
inbox = get_actor_inbox(actor)
url = urlsplit(inbox)
# XXX: Digest
data = json.dumps(message)
headers = {
'(request-target)': 'post {}'.format(url.path),
'Content-Length': str(len(data)),
'Content-Type': 'application/activity+json',
'User-Agent': 'ActivityRelay',
'Host': url.netloc,
'Digest': 'SHA-256={}'.format(generate_body_digest(data)),
'Date': datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
}
headers['signature'] = sign_headers(headers, PRIVKEY, our_key_id)
headers.pop('(request-target)')
headers.pop('Host')
logging.debug('%r >> %r', inbox, message)
global sem
async with sem:
try:
async with aiohttp.ClientSession(trace_configs=[http_debug()]) as session:
async with session.post(inbox, data=data, headers=headers) as resp:
if resp.status == 202:
return
resp_payload = await resp.text()
logging.debug('%r >> resp %r', inbox, resp_payload)
except Exception as e:
logging.info('Caught %r while pushing to %r.', e, inbox)
async def fetch_nodeinfo(domain):
headers = {'Accept': 'application/json'}
nodeinfo_url = None
wk_nodeinfo = await fetch_actor(f'https://{domain}/.well-known/nodeinfo', headers=headers)
if not wk_nodeinfo:
return
for link in wk_nodeinfo.get('links', ''):
if link['rel'] == 'http://nodeinfo.diaspora.software/ns/schema/2.0':
nodeinfo_url = link['href']
break
if not nodeinfo_url:
return
nodeinfo_data = await fetch_actor(nodeinfo_url, headers=headers)
software = nodeinfo_data.get('software')
return software.get('name') if software else None
async def follow_remote_actor(actor_uri):
actor = await fetch_actor(actor_uri)
if not actor:
logging.info('failed to fetch actor at: %r', actor_uri)
return
if AP_CONFIG['whitelist_enabled'] is True and urlsplit(actor_uri).hostname not in AP_CONFIG['whitelist']:
logging.info('refusing to follow non-whitelisted actor: %r', actor_uri)
return
logging.info('following: %r', actor_uri)
message = {
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Follow",
"to": [actor['id']],
"object": actor['id'],
"id": "https://{}/activities/{}".format(AP_CONFIG['host'], uuid.uuid4()),
"actor": "https://{}/actor".format(AP_CONFIG['host'])
}
await push_message_to_actor(actor, message, "https://{}/actor#main-key".format(AP_CONFIG['host']))
async def unfollow_remote_actor(actor_uri):
actor = await fetch_actor(actor_uri)
if not actor:
logging.info('failed to fetch actor at: %r', actor_uri)
return
logging.info('unfollowing: %r', actor_uri)
message = {
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Undo",
"to": [actor['id']],
"object": {
"type": "Follow",
"object": actor_uri,
"actor": actor['id'],
"id": "https://{}/activities/{}".format(AP_CONFIG['host'], uuid.uuid4())
},
"id": "https://{}/activities/{}".format(AP_CONFIG['host'], uuid.uuid4()),
"actor": "https://{}/actor".format(AP_CONFIG['host'])
}
await push_message_to_actor(actor, message, "https://{}/actor#main-key".format(AP_CONFIG['host']))
tag_re = re.compile(r'(<!--.*?-->|<[^>]*>)')
def strip_html(data):
no_tags = tag_re.sub('', data)
return cgi.escape(no_tags)
def distill_inboxes(actor, object_id):
global DATABASE
origin_hostname = urlsplit(object_id).hostname
inbox = get_actor_inbox(actor)
targets = [target for target in DATABASE.get('relay-list', []) if target != inbox]
targets = [target for target in targets if urlsplit(target).hostname != origin_hostname]
hostnames = [urlsplit(target).hostname for target in targets]
assert inbox not in targets
assert origin_hostname not in hostnames
return targets
def distill_object_id(activity):
logging.debug('>> determining object ID for %r', activity['object'])
obj = activity['object']
if isinstance(obj, str):
return obj
return obj['id']
async def handle_relay(actor, data, request):
global CACHE
object_id = distill_object_id(data)
if object_id in CACHE:
logging.debug('>> already relayed %r as %r', object_id, CACHE[object_id])
return
activity_id = "https://{}/activities/{}".format(request.host, uuid.uuid4())
message = {
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Announce",
"to": ["https://{}/followers".format(request.host)],
"actor": "https://{}/actor".format(request.host),
"object": object_id,
"id": activity_id
}
logging.debug('>> relay: %r', message)
inboxes = distill_inboxes(actor, object_id)
futures = [push_message_to_actor({'inbox': inbox}, message, 'https://{}/actor#main-key'.format(request.host)) for inbox in inboxes]
asyncio.ensure_future(asyncio.gather(*futures))
CACHE[object_id] = activity_id
async def handle_forward(actor, data, request):
object_id = distill_object_id(data)
logging.debug('>> Relay %r', data)
inboxes = distill_inboxes(actor, object_id)
futures = [
push_message_to_actor(
{'inbox': inbox},
data,
'https://{}/actor#main-key'.format(request.host))
for inbox in inboxes]
asyncio.ensure_future(asyncio.gather(*futures))
async def handle_follow(actor, data, request):
global DATABASE
following = DATABASE.get('relay-list', [])
inbox = get_actor_inbox(actor)
if urlsplit(inbox).hostname in AP_CONFIG['blocked_instances']:
return
if inbox not in following:
following += [inbox]
DATABASE['relay-list'] = following
asyncio.ensure_future(follow_remote_actor(actor['id']))
message = {
"@context": "https://www.w3.org/ns/activitystreams",
"type": "Accept",
"to": [actor["id"]],
"actor": "https://{}/actor".format(request.host),
# this is wrong per litepub, but mastodon < 2.4 is not compliant with that profile.
"object": {
"type": "Follow",
"id": data["id"],
"object": "https://{}/actor".format(request.host),
"actor": actor["id"]
},
"id": "https://{}/activities/{}".format(request.host, uuid.uuid4()),
}
asyncio.ensure_future(push_message_to_actor(actor, message, 'https://{}/actor#main-key'.format(request.host)))
async def handle_undo(actor, data, request):
global DATABASE
child = data['object']
if child['type'] == 'Follow':
following = DATABASE.get('relay-list', [])
inbox = get_actor_inbox(actor)
if inbox in following:
following.remove(inbox)
DATABASE['relay-list'] = following
await unfollow_remote_actor(actor['id'])
processors = {
'Announce': handle_relay,
'Create': handle_relay,
'Delete': handle_forward,
'Follow': handle_follow,
'Undo': handle_undo,
'Update': handle_forward,
}
async def inbox(request):
data = await request.json()
instance = urlsplit(data['actor']).hostname
if AP_CONFIG['blocked_software']:
software = await fetch_nodeinfo(instance)
if software and software.lower() in AP_CONFIG['blocked_software']:
raise aiohttp.web.HTTPUnauthorized(body='relays have been blocked', content_type='text/plain')
if 'actor' not in data or not request['validated']:
raise aiohttp.web.HTTPUnauthorized(body='access denied', content_type='text/plain')
elif data['type'] != 'Follow' and 'https://{}/inbox'.format(instance) not in DATABASE['relay-list']:
raise aiohttp.web.HTTPUnauthorized(body='access denied', content_type='text/plain')
elif AP_CONFIG['whitelist_enabled'] is True and instance not in AP_CONFIG['whitelist']:
raise aiohttp.web.HTTPUnauthorized(body='access denied', content_type='text/plain')
actor = await fetch_actor(data["actor"])
actor_uri = 'https://{}/actor'.format(request.host)
logging.debug(">> payload %r", data)
processor = processors.get(data['type'], None)
if processor:
await processor(actor, data, request)
return aiohttp.web.Response(body=b'{}', content_type='application/activity+json')
app.router.add_post('/inbox', inbox)

272
relay/config.py Normal file
View file

@ -0,0 +1,272 @@
import json
import yaml
from pathlib import Path
from urllib.parse import urlparse
relay_software_names = [
'activityrelay',
'aoderelay',
'social.seattle.wa.us-relay',
'unciarelay'
]
class DotDict(dict):
def __getattr__(self, k):
try:
return self[k]
except KeyError:
raise AttributeError(f'{self.__class__.__name__} object has no attribute {k}') from None
def __setattr__(self, k, v):
try:
if k in self._ignore_keys:
super().__setattr__(k, v)
except AttributeError:
pass
if k.startswith('_'):
super().__setattr__(k, v)
else:
self[k] = v
def __setitem__(self, k, v):
if type(v) == dict:
v = DotDict(v)
super().__setitem__(k, v)
def __delattr__(self, k):
try:
dict.__delitem__(self, k)
except KeyError:
raise AttributeError(f'{self.__class__.__name__} object has no attribute {k}') from None
class RelayConfig(DotDict):
apkeys = {
'host',
'whitelist_enabled',
'blocked_software',
'blocked_instances',
'whitelist'
}
cachekeys = {
'json',
'objects',
'digests'
}
def __init__(self, path, is_docker):
if is_docker:
path = '/data/relay.yaml'
self._isdocker = is_docker
self._path = Path(path).expanduser()
super().__init__({
'db': str(self._path.parent.joinpath(f'{self._path.stem}.jsonld')),
'listen': '0.0.0.0',
'port': 8080,
'note': 'Make a note about your instance here.',
'push_limit': 512,
'host': 'relay.example.com',
'blocked_software': [],
'blocked_instances': [],
'whitelist': [],
'whitelist_enabled': False,
'json': 1024,
'objects': 1024,
'digests': 1024
})
def __setitem__(self, key, value):
if self._isdocker and key in ['db', 'listen', 'port']:
return
if key in ['blocked_instances', 'blocked_software', 'whitelist']:
assert isinstance(value, (list, set, tuple))
elif key in ['port', 'json', 'objects', 'digests']:
assert isinstance(value, (int))
elif key == 'whitelist_enabled':
assert isinstance(value, bool)
super().__setitem__(key, value)
@property
def db(self):
return Path(self['db']).expanduser().resolve()
@property
def path(self):
return self._path
@property
def actor(self):
return f'https://{self.host}/actor'
@property
def inbox(self):
return f'https://{self.host}/inbox'
@property
def keyid(self):
return f'{self.actor}#main-key'
def ban_instance(self, instance):
if instance.startswith('http'):
instance = urlparse(instance).hostname
if self.is_banned(instance):
return False
self.blocked_instances.append(instance)
return True
def unban_instance(self, instance):
if instance.startswith('http'):
instance = urlparse(instance).hostname
try:
self.blocked_instances.remove(instance)
return True
except:
return False
def ban_software(self, software):
if self.is_banned_software(software):
return False
self.blocked_software.append(software)
return True
def unban_software(self, software):
try:
self.blocked_software.remove(software)
return True
except:
return False
def add_whitelist(self, instance):
if instance.startswith('http'):
instance = urlparse(instance).hostname
if self.is_whitelisted(instance):
return False
self.whitelist.append(instance)
return True
def del_whitelist(self, instance):
if instance.startswith('http'):
instance = urlparse(instance).hostname
try:
self.whitelist.remove(instance)
return True
except:
return False
def is_banned(self, instance):
if instance.startswith('http'):
instance = urlparse(instance).hostname
return instance in self.blocked_instances
def is_banned_software(self, software):
if not software:
return False
return software.lower() in self.blocked_software
def is_whitelisted(self, instance):
if instance.startswith('http'):
instance = urlparse(instance).hostname
return instance in self.whitelist
def load(self):
options = {}
try:
options['Loader'] = yaml.FullLoader
except AttributeError:
pass
try:
with open(self.path) as fd:
config = yaml.load(fd, **options)
except FileNotFoundError:
return False
if not config:
return False
for key, value in config.items():
if key in ['ap', 'cache']:
for k, v in value.items():
if k not in self:
continue
self[k] = v
elif key not in self:
continue
self[key] = value
if self.host.endswith('example.com'):
return False
return True
def save(self):
config = {
'db': self['db'],
'listen': self.listen,
'port': self.port,
'note': self.note,
'push_limit': self.push_limit,
'ap': {key: self[key] for key in self.apkeys},
'cache': {key: self[key] for key in self.cachekeys}
}
with open(self._path, 'w') as fd:
yaml.dump(config, fd, sort_keys=False)
return config

View file

@ -1,43 +1,121 @@
import asyncio
import json
import logging
import urllib.parse
import simplejson as json
from sys import exit
import traceback
from Crypto.PublicKey import RSA
from urllib.parse import urlparse
from . import CONFIG
AP_CONFIG = CONFIG['ap']
try:
with open(CONFIG['db']) as f:
DATABASE = json.load(f)
except FileNotFoundError:
logging.info('No database was found, making a new one.')
DATABASE = {}
except json.decoder.JSONDecodeError:
logging.info('Invalid JSON in db. Exiting...')
exit(1)
following = DATABASE.get('relay-list', [])
for inbox in following:
if urllib.parse.urlsplit(inbox).hostname in AP_CONFIG['blocked_instances']:
following.remove(inbox)
elif AP_CONFIG['whitelist_enabled'] is True and urllib.parse.urlsplit(inbox).hostname not in AP_CONFIG['whitelist']:
following.remove(inbox)
DATABASE['relay-list'] = following
if 'actors' in DATABASE:
DATABASE.pop('actors')
async def database_save():
while True:
with open(CONFIG['db'], 'w') as f:
json.dump(DATABASE, f)
await asyncio.sleep(30)
class RelayDatabase:
def __init__(self, config):
self.config = config
self.data = None
self.PRIVKEY = None
asyncio.ensure_future(database_save())
@property
def PUBKEY(self):
return self.PRIVKEY.publickey()
@property
def pubkey(self):
return self.PUBKEY.exportKey('PEM').decode('utf-8')
@property
def privkey(self):
try:
return self.data['private-key']
except KeyError:
return False
@property
def hostnames(self):
return [urlparse(inbox).hostname for inbox in self.inboxes]
@property
def inboxes(self):
return self.data.get('relay-list', [])
def generate_key(self):
self.PRIVKEY = RSA.generate(4096)
self.data['private-key'] = self.PRIVKEY.exportKey('PEM').decode('utf-8')
def load(self):
new_db = True
try:
with self.config.db.open() as fd:
self.data = json.load(fd)
key = self.data.pop('actorKeys', None)
if key:
self.data['private-key'] = key.get('privateKey')
self.data.pop('actors', None)
new_db = False
except FileNotFoundError:
pass
except json.decoder.JSONDecodeError as e:
if self.config.db.stat().st_size > 0:
raise e from None
if not self.data:
logging.info('No database was found. Making a new one.')
self.data = {}
for inbox in self.inboxes:
if self.config.is_banned(inbox) or (self.config.whitelist_enabled and not self.config.is_whitelisted(inbox)):
self.del_inbox(inbox)
if not self.privkey:
logging.info("No actor keys present, generating 4096-bit RSA keypair.")
self.generate_key()
else:
self.PRIVKEY = RSA.importKey(self.privkey)
self.save()
return not new_db
def save(self):
with self.config.db.open('w') as fd:
data = {
'relay-list': self.inboxes,
'private-key': self.privkey
}
json.dump(data, fd, indent=4)
def get_inbox(self, domain):
if domain.startswith('http'):
domain = urlparse(domain).hostname
for inbox in self.inboxes:
if domain == urlparse(inbox).hostname:
return inbox
def add_inbox(self, inbox):
assert inbox.startswith('https')
assert inbox not in self.inboxes
self.data['relay-list'].append(inbox)
def del_inbox(self, inbox):
if inbox not in self.inboxes:
raise KeyError(inbox)
self.data['relay-list'].remove(inbox)

View file

@ -1,36 +0,0 @@
import aiohttp.web
import urllib.parse
from . import app, CONFIG
from .database import DATABASE
host = CONFIG['ap']['host']
note = CONFIG['note']
inboxes = DATABASE.get('relay-list', [])
async def default(request):
targets = '<br>'.join([urllib.parse.urlsplit(target).hostname for target in inboxes])
return aiohttp.web.Response(
status=200,
content_type="text/html",
charset="utf-8",
text="""
<html><head>
<title>ActivityPub Relay at {host}</title>
<style>
p {{ color: #FFFFFF; font-family: monospace, arial; font-size: 100%; }}
body {{ background-color: #000000; }}
</style>
</head>
<body>
<p>This is an Activity Relay for fediverse instances.</p>
<p>{note}</p>
<p>For Mastodon and Misskey instances, you may subscribe to this relay with the address: <a href="https://{host}/inbox">https://{host}/inbox</a></p>
<p>For Pleroma and other instances, you may subscribe to this relay with the address: <a href="https://{host}/actor">https://{host}/actor</a></p>
<p>To host your own relay, you may download the code at this address: <a href="https://git.pleroma.social/pleroma/relay">https://git.pleroma.social/pleroma/relay</a></p>
<br><p>List of {count} registered instances:<br>{targets}</p>
</body></html>
""".format(host=host, note=note,targets=targets,count=len(inboxes)))
app.router.add_get('/', default)

View file

@ -1,6 +1,5 @@
import logging
import aiohttp
import aiohttp.web
from collections import defaultdict
@ -59,8 +58,11 @@ async def on_request_exception(session, trace_config_ctx, params):
def http_debug():
if logging.DEBUG >= logging.root.level:
return
trace_config = aiohttp.TraceConfig()
trace_config.on_request_start.append(on_request_start)