Compare commits

..

7 commits

Author SHA1 Message Date
Izalia Mae 4acdfdbfc1 update docs 2022-12-20 08:00:18 -05:00
Izalia Mae 261dce50ab let setup command configure the database 2022-12-20 07:59:58 -05:00
Izalia Mae ed25fcab35 remove print call 2022-12-20 06:17:20 -05:00
Izalia Mae 8eb60cb0f4 split database into sub-module 2022-12-20 06:09:27 -05:00
Izalia Mae be556163c9 only set signal handler on server start and stop 2022-12-20 06:08:17 -05:00
Izalia Mae 4979d598f1 call app.setup first 2022-12-20 06:05:06 -05:00
Izalia Mae 04ae6a8851 remove appdirs dep and add option to set sqlite database path 2022-12-20 06:01:20 -05:00
12 changed files with 389 additions and 211 deletions

View file

@ -3,9 +3,9 @@
There are a number of commands to manage your relay's database and config. You can add `--help` to There are a number of commands to manage your relay's database and config. You can add `--help` to
any category or command to get help on that specific option (ex. `activityrelay inbox --help`). any category or command to get help on that specific option (ex. `activityrelay inbox --help`).
Note: Unless specified, it is recommended to run any commands while the relay is shutdown. A config file can be specified by adding `--config [path/to/config.yaml]`.
Note 2: `activityrelay` is only available via pip or pipx if `~/.local/bin` is in `$PATH`. If it Note: `activityrelay` is only available via pip or pipx if `~/.local/bin` is in `$PATH`. If it
isn't, use `python3 -m relay` if installed via pip or `~/.local/bin/activityrelay` if installed isn't, use `python3 -m relay` if installed via pip or `~/.local/bin/activityrelay` if installed
via pipx via pipx
@ -24,26 +24,35 @@ Run the setup wizard to configure your relay.
activityrelay setup activityrelay setup
## Convert
Convert an old `relay.yaml` and `relay.jsonld` to the newer formats.
activityrelay convert [--old-config relay.yaml]
## Config ## Config
Manage the relay config Manage the relay config.
activityrelay config activityrelay config
### List ### List
List the current config key/value pairs List the current config key/value pairs.
activityrelay config list activityrelay config list
### Set ### Set
Set a value for a config option Set a value for a config option.
activityrelay config set <key> <value> activityrelay config set <key> <value>
note: The relay must be restarted if setting `log_level`, `workers`, `push_limit`, or `http_timeout`
## Inbox ## Inbox
@ -92,6 +101,32 @@ not exist anymore, use the `inbox remove` command instead.
Note: The relay must be running for this command to work. Note: The relay must be running for this command to work.
## Request
Manage instance follow requests.
### List
List all instances asking to follow the relay.
activityrelay request list
### Approve
Allow an instance to join the relay.
activityrelay request approve <domain>
### Deny
Disallow an instance to join the relay.
activityrelay request deny <domain>
## Whitelist ## Whitelist
Manage the whitelisted domains. Manage the whitelisted domains.
@ -120,7 +155,7 @@ Remove a domain from the whitelist.
### Import ### Import
Add all current inboxes to the whitelist Add all current inboxes to the whitelist.
activityrelay whitelist import activityrelay whitelist import
@ -132,7 +167,7 @@ Manage the instance ban list.
### List ### List
List the currently banned instances List the currently banned instances.
activityrelay instance list activityrelay instance list

View file

@ -2,14 +2,6 @@
## General ## General
### DB
The path to the database. It contains the relay actor private key and all subscribed
instances. If the path is not absolute, it is relative to the working directory.
db: relay.jsonld
### Listener ### Listener
The address and port the relay will listen on. If the reverse proxy (nginx, apache, caddy, etc) The address and port the relay will listen on. If the reverse proxy (nginx, apache, caddy, etc)
@ -19,46 +11,6 @@ is running on the same host, it is recommended to change `listen` to `localhost`
port: 8080 port: 8080
### Note
A small blurb to describe your relay instance. This will show up on the relay's home page.
note: "Make a note about your instance here."
### Post Limit
The maximum number of messages to send out at once. For each incoming message, a message will be
sent out to every subscribed instance minus the instance which sent the message. This limit
is to prevent too many outgoing connections from being made, so adjust if necessary.
Note: If the `workers` option is set to anything above 0, this limit will be per worker.
push_limit: 512
### Push Workers
The relay can be configured to use threads to push messages out. For smaller relays, this isn't
necessary, but bigger ones (>100 instances) will want to set this to the number of available cpu
threads.
workers: 0
### JSON GET cache limit
JSON objects (actors, nodeinfo, etc) will get cached when fetched. This will set the max number of
objects to keep in the cache.
json_cache: 1024
## AP
Various ActivityPub-related settings
### Host ### Host
The domain your relay will use to identify itself. The domain your relay will use to identify itself.
@ -66,40 +18,123 @@ The domain your relay will use to identify itself.
host: relay.example.com host: relay.example.com
### Whitelist Enabled ## Database
If set to `true`, only instances in the whitelist can follow the relay. Any subscribed instances ### Type
not in the whitelist will be removed from the inbox list on startup.
whitelist_enabled: false The type of SQL database to use. Options:
* sqlite (default)
* postgresql
* mysql
type: sqlite
### Whitelist ### Minimum Connections
A list of domains of instances which are allowed to subscribe to your relay. The minimum number of database connections to keep open (does nothing at the moment)
whitelist: min_connections: 0
- bad-instance.example.com
- another-bad-instance.example.com
### Blocked Instances ### Maximum Connections
A list of instances which are unable to follow the instance. If a subscribed instance is added to The maximum number of database connections to open (does nothing at the moment)
the block list, it will be removed from the inbox list on startup.
blocked_instances: max_connections: 10
- bad-instance.example.com
- another-bad-instance.example.com
### Blocked Software ## Sqlite
A list of ActivityPub software which cannot follow your relay. This list is empty by default, but ### Database
setting this to the below list will block all other relays and prevent relay chains
blocked_software: The path to the database file.
- activityrelay
- aoderelay database: relay.sqlite3
- social.seattle.wa.us-relay
- unciarelay If the path is relative, it will be relative to the directory the config file is located. For
instance, if the config is located at `/home/izalia/.config/activityrelay/config.yaml`, the
following:
relay.sqlite3
will resolve to:
/home/izalia/.config/activityrelay/relay.sqlite3
## PostgreSQL
### Database
Name of the database to use.
database: activityrelay
### Hostname
The address to use when connecting to the database. A value of `null` will use the default of
`/var/run/postgresql`
hostname: null
### Port
The port to use when connecting to the database. A value of `null` will use the default of `5432`
port: null
### Username
The user to use when connecting to the database. A value of `null` will use the current system
username.
username: null
### Password
The password for the database user.
password: null
## MySQL
### Database
Name of the database to use.
database: activityrelay
### Hostname
The address to use when connecting to the database. A value of `null` will use the default of
`/var/run/mysqld/mysqld.sock`
### Port
The port to use when connecting to the database. A value of `null` will use the default of `3306`
port: null
### Username
The user to use when connecting to the database. A value of `null` will use the current system
username.
username: null
### Password
The password for the database user.
password: null

View file

@ -5,6 +5,19 @@ proxy, and setup the relay to run via a supervisor. Example configs for caddy, n
in `installation/` in `installation/`
## Pre-build Executables
All in one executables can be downloaded from `https://git.pleroma.social/pleroma/relay/-/releases`
under the `Other` section of `Assets`. They don't require any extra setup and can be placed
anywhere. Run the setup wizard
./activityrelay setup
and start it up when done
./activityrelay run
## Pipx ## Pipx
Pipx uses pip and a custom venv implementation to automatically install modules into a Python Pipx uses pip and a custom venv implementation to automatically install modules into a Python

View file

@ -1,9 +1,9 @@
general: general:
# Address the relay will listen on. Set to "0.0.0.0" for any address # Address the relay will listen on. Set to "0.0.0.0" for any address
listen: 0.0.0.0 listen: 0.0.0.0
# Port the relay will listen on # TCP port the relay will listen on
port: 3621 port: 3621
# Domain the relay will advertise as # Domain the relay will advertise itself as
host: relay.example.com host: relay.example.com
database: database:
@ -14,6 +14,9 @@ database:
# Maximum number of database connections to open # Maximum number of database connections to open
max_connections: 10 max_connections: 10
sqlite:
database: relay.sqlite3
postgres: postgres:
database: activityrelay database: activityrelay
hostname: null hostname: null

View file

@ -40,7 +40,6 @@ class Application(web.Application):
self['last_worker'] = 0 self['last_worker'] = 0
self.database.create() self.database.create()
self.set_signal_handler()
with self.database.session as s: with self.database.session as s:
set_level(s.get_config('log_level')) set_level(s.get_config('log_level'))
@ -98,10 +97,10 @@ class Application(web.Application):
self['last_worker'] = 0 self['last_worker'] = 0
def set_signal_handler(self): def set_signal_handler(self, enable=True):
for sig in {'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGTERM'}: for sig in {'SIGHUP', 'SIGINT', 'SIGQUIT', 'SIGTERM'}:
try: try:
signal.signal(getattr(signal, sig), self.stop) signal.signal(getattr(signal, sig), self.stop if enable else signal.SIG_DFL)
# some signals don't exist in windows, so skip them # some signals don't exist in windows, so skip them
except AttributeError: except AttributeError:
@ -129,6 +128,7 @@ class Application(web.Application):
async def handle_run(self): async def handle_run(self):
self.set_signal_handler(True)
self['running'] = True self['running'] = True
with self.database.session as s: with self.database.session as s:
@ -161,6 +161,7 @@ class Application(web.Application):
self['starttime'] = None self['starttime'] = None
self['running'] = False self['running'] = False
self['workers'].clear() self['workers'].clear()
self.set_signal_handler(False)
class PushWorker(threading.Thread): class PushWorker(threading.Thread):

View file

@ -1,10 +1,10 @@
import appdirs
import os import os
import sys import sys
import yaml import yaml
from functools import cached_property from functools import cached_property
from pathlib import Path from pathlib import Path
from platform import system
from .misc import AppBase, DotDict from .misc import AppBase, DotDict
@ -16,6 +16,7 @@ DEFAULTS = {
'database_type': 'sqlite', 'database_type': 'sqlite',
'database_min_connections': 0, 'database_min_connections': 0,
'database_max_connections': 10, 'database_max_connections': 10,
'sqlite_database': Path('relay.sqlite3'),
'postgres_database': 'activityrelay', 'postgres_database': 'activityrelay',
'postgres_hostname': None, 'postgres_hostname': None,
'postgres_port': None, 'postgres_port': None,
@ -31,30 +32,40 @@ DEFAULTS = {
CATEGORY_NAMES = [ CATEGORY_NAMES = [
'general', 'general',
'database', 'database',
'sqlite',
'postgres', 'postgres',
'mysql' 'mysql'
] ]
CONFIG_DIRS = [
Path.cwd(),
Path(appdirs.user_config_dir('activityrelay'))
]
def get_config_dir(): def get_config_dir():
for path in CONFIG_DIRS: cwd = Path.cwd().joinpath('config.yaml')
cfgpath = path.joinpath('config.yaml') plat = system()
if cwd.exists():
return cwd
elif plat == 'Linux':
cfgpath = Path('~/.config/activityrelay/config.yaml').expanduser()
if cfgpath.exists(): if cfgpath.exists():
return cfgpath return cfgpath
if sys.platform == 'linux':
etcpath = Path('/etc/activityrelay/config.yaml') etcpath = Path('/etc/activityrelay/config.yaml')
if etcpath.exists(): if etcpath.exists() and os.getuid() == etcpath.stat().st_uid:
return etcpath
elif plat == 'Windows':
cfgpath = Path('~/AppData/Roaming/activityrelay/config.yaml').expanduer()
if cfgpath.exists():
return cfgpath return cfgpath
return Path.cwd().joinpath('config.yaml') elif plat == 'Darwin':
cfgpath = Path('~/Library/Application Support/activityaelay/config.yaml')
return cwd
class Config(AppBase, dict): class Config(AppBase, dict):
@ -68,19 +79,26 @@ class Config(AppBase, dict):
path = get_config_dir() path = get_config_dir()
else: else:
path = Path(path).expanduser().resolve() path = Path(path).expanduser()
self._path = path self._path = path
self.load() self.load()
def __setitem__(self, key, value): def __setitem__(self, key, value):
if key in {'database', 'hostname', 'port', 'username', 'password'}:
key = f'{self.dbtype}_{key}'
if (self.is_docker and key in {'general_host', 'general_port'}) or value == '__DEFAULT__': if (self.is_docker and key in {'general_host', 'general_port'}) or value == '__DEFAULT__':
value = DEFAULTS[key] value = DEFAULTS[key]
elif key in {'general_port', 'database_min_connections', 'database_max_connections'}: elif key in {'general_port', 'database_min_connections', 'database_max_connections'}:
value = int(value) value = int(value)
elif key == 'sqlite_database':
if not isinstance(value, Path):
value = Path(value)
dict.__setitem__(self, key, value) dict.__setitem__(self, key, value)
@ -93,7 +111,11 @@ class Config(AppBase, dict):
} }
if self.dbtype == 'sqlite': if self.dbtype == 'sqlite':
config['database'] = self.path.with_name('relay.sqlite3') if not self['sqlite_database'].is_absolute():
config['database'] = self.path.with_name(str(self['sqlite_database'])).resolve()
else:
config['database'] = self['sqlite_database'].resolve()
else: else:
for key, value in self.items(): for key, value in self.items():

View file

@ -0,0 +1,17 @@
import tinysql
from .base import DEFAULT_CONFIG, RELAY_SOFTWARE, TABLES
from .connection import Connection
from .rows import ROWS
class Database(tinysql.Database):
def __init__(self, **config):
tinysql.Database.__init__(self, **config,
connection_class = Connection,
row_classes = ROWS
)
def create(self):
self.create_database(TABLES)

67
relay/database/base.py Normal file
View file

@ -0,0 +1,67 @@
from tinysql import Column, Table
DEFAULT_CONFIG = {
'description': ('str', 'Make a note about your relay here'),
'http_timeout': ('int', 10),
'json_cache': ('int', 1024),
'log_level': ('str', 'INFO'),
'name': ('str', 'ActivityRelay'),
'privkey': ('str', ''),
'push_limit': ('int', 512),
'require_approval': ('bool', False),
'version': ('int', 20221211),
'whitelist': ('bool', False),
'workers': ('int', 8)
}
RELAY_SOFTWARE = [
'activity-relay', # https://github.com/yukimochi/Activity-Relay
'activityrelay', # https://git.pleroma.social/pleroma/relay
'aoderelay', # https://git.asonix.dog/asonix/relay
'feditools-relay' # https://git.ptzo.gdn/feditools/relay
]
TABLES = [
Table('config',
Column('key', 'text', unique=True, nullable=False, primary_key=True),
Column('value', 'text')
),
Table('instances',
Column('id', 'serial'),
Column('domain', 'text', unique=True, nullable=False),
Column('actor', 'text'),
Column('inbox', 'text', nullable=False),
Column('followid', 'text'),
Column('software', 'text'),
Column('note', 'text'),
Column('joined', 'datetime', nullable=False),
Column('updated', 'datetime')
),
Table('whitelist',
Column('id', 'serial'),
Column('domain', 'text', unique=True),
Column('created', 'datetime', nullable=False)
),
Table('bans',
Column('id', 'serial'),
Column('name', 'text', unique=True),
Column('note', 'text'),
Column('type', 'text', nullable=False),
Column('created', 'datetime', nullable=False)
),
Table('users',
Column('id', 'serial'),
Column('handle', 'text', unique=True, nullable=False),
Column('domain', 'text', nullable=False),
Column('api_token', 'text'),
Column('created', 'datetime', nullable=False),
Column('updated', 'datetime')
),
Table('tokens',
Column('id', 'text', unique=True, nullable=False, primary_key=True),
Column('userid', 'integer', nullable=False),
Column('created', 'datetime', nullable=False),
Column('updated', 'datetime')
)
]

View file

@ -1,91 +1,10 @@
import tinysql import tinysql
from datetime import datetime from datetime import datetime
from tinysql import Column, Table
from urllib.parse import urlparse from urllib.parse import urlparse
from .logger import set_level from .base import DEFAULT_CONFIG
from .misc import AppBase, DotDict, boolean from ..misc import DotDict
TABLES = [
Table('config',
Column('key', 'text', unique=True, nullable=False, primary_key=True),
Column('value', 'text')
),
Table('instances',
Column('id', 'serial'),
Column('domain', 'text', unique=True, nullable=False),
Column('actor', 'text'),
Column('inbox', 'text', nullable=False),
Column('followid', 'text'),
Column('software', 'text'),
Column('actor_data', 'json'),
Column('note', 'text'),
Column('joined', 'datetime', nullable=False),
Column('updated', 'datetime')
),
Table('whitelist',
Column('id', 'serial'),
Column('domain', 'text', unique=True),
Column('created', 'datetime', nullable=False)
),
Table('bans',
Column('id', 'serial'),
Column('name', 'text', unique=True),
Column('note', 'text'),
Column('type', 'text', nullable=False),
Column('created', 'datetime', nullable=False)
),
Table('users',
Column('id', 'serial'),
Column('handle', 'text', unique=True, nullable=False),
Column('domain', 'text', nullable=False),
Column('api_token', 'text'),
Column('created', 'datetime', nullable=False),
Column('updated', 'datetime')
),
Table('tokens',
Column('id', 'text', unique=True, nullable=False, primary_key=True),
Column('userid', 'integer', nullable=False),
Column('created', 'datetime', nullable=False),
Column('updated', 'datetime')
)
]
DEFAULT_CONFIG = {
'description': ('str', 'Make a note about your relay here'),
'http_timeout': ('int', 10),
'json_cache': ('int', 1024),
'log_level': ('str', 'INFO'),
'name': ('str', 'ActivityRelay'),
'privkey': ('str', ''),
'push_limit': ('int', 512),
'require_approval': ('bool', False),
'version': ('int', 20221211),
'whitelist': ('bool', False),
'workers': ('int', 8)
}
RELAY_SOFTWARE = [
'activityrelay', # https://git.pleroma.social/pleroma/relay
'aoderelay', # https://git.asonix.dog/asonix/relay
'feditools-relay' # https://git.ptzo.gdn/feditools/relay
]
class Database(AppBase, tinysql.Database):
def __init__(self, **config):
tinysql.Database.__init__(self, **config,
connection_class = Connection,
row_classes = [
ConfigRow
]
)
def create(self):
self.create_database(TABLES)
class Connection(tinysql.ConnectionMixin): class Connection(tinysql.ConnectionMixin):
@ -163,12 +82,12 @@ class Connection(tinysql.ConnectionMixin):
if not row: if not row:
return DEFAULT_CONFIG[key][1] return DEFAULT_CONFIG[key][1]
return row.get_value() return row.value
def get_config_all(self): def get_config_all(self):
rows = self.select('config').all() rows = self.select('config').all()
config = DotDict({row.key: row.get_value() for row in rows}) config = DotDict({row.key: row.value for row in rows})
for key, data in DEFAULT_CONFIG.items(): for key, data in DEFAULT_CONFIG.items():
if key not in config: if key not in config:
@ -247,8 +166,8 @@ class Connection(tinysql.ConnectionMixin):
if value == '__DEFAULT__': if value == '__DEFAULT__':
value = DEFAULT_CONFIG[key][1] value = DEFAULT_CONFIG[key][1]
if key == 'log_level': elif key == 'log_level' and not getattr(logging, value.upper(), False):
set_level(value) raise KeyError(value)
row = self.select('config', key=key).one() row = self.select('config', key=key).one()
@ -318,24 +237,3 @@ class Connection(tinysql.ConnectionMixin):
'domain': domain, 'domain': domain,
'created': datetime.now() 'created': datetime.now()
}) })
class ConfigRow(tinysql.Row):
__table__ = 'config'
def get_value(self):
type = DEFAULT_CONFIG[self.key][0]
if type == 'int':
return int(self.value)
elif type == 'bool':
return boolean(self.value.encode('utf-8'))
elif type == 'list':
return json.loads(value)
elif type == 'json':
return DotDict.parse(value)
return self.value

35
relay/database/rows.py Normal file
View file

@ -0,0 +1,35 @@
import json
from tinysql import Row
from .base import DEFAULT_CONFIG
from ..misc import DotDict, boolean
ROWS = []
def register(cls):
ROWS.append(cls)
return cls
@register
class ConfigRow(Row):
__table__ = 'config'
@property
def value(self):
type = DEFAULT_CONFIG[self.key][0]
if type == 'int':
return int(self['value'])
elif type == 'bool':
return boolean(self['value'])
elif type == 'list':
return json.loads(self['value'])
elif type == 'json':
return DotDict.parse(self['value'])
return self['value']

View file

@ -31,6 +31,9 @@ def cli(ctx, config):
global app global app
app = Application(config) app = Application(config)
if ctx.invoked_subcommand != 'convert':
app.setup()
if not ctx.invoked_subcommand: if not ctx.invoked_subcommand:
if app.config.host.endswith('example.com'): if app.config.host.endswith('example.com'):
cli_setup.callback() cli_setup.callback()
@ -38,13 +41,12 @@ def cli(ctx, config):
else: else:
cli_run.callback() cli_run.callback()
if ctx.invoked_subcommand != 'convert':
app.setup()
@cli.command('convert') @cli.command('convert')
@click.option('--old-config', '-o', help='path to the old relay config') @click.option('--old-config', '-o', help='path to the old relay config')
def cli_convert(old_config): def cli_convert(old_config):
'Convert an old relay.yaml and relay.jsonld to the the new formats'
with open(old_config or 'relay.yaml') as fd: with open(old_config or 'relay.yaml') as fd:
config = yaml.load(fd.read(), Loader=yaml.SafeLoader) config = yaml.load(fd.read(), Loader=yaml.SafeLoader)
ap = config.get('ap', {}) ap = config.get('ap', {})
@ -106,7 +108,10 @@ def cli_setup():
'Generate a new config' 'Generate a new config'
while True: while True:
app.config['general_host'] = click.prompt('What domain will the relay be hosted on?', default=app.config.host) app.config['general_host'] = click.prompt(
'What domain will the relay be hosted on?',
default = app.config.host
)
if not app.config.host.endswith('example.com'): if not app.config.host.endswith('example.com'):
break break
@ -114,12 +119,60 @@ def cli_setup():
click.echo('The domain must not be example.com') click.echo('The domain must not be example.com')
if not app.config.is_docker: if not app.config.is_docker:
app.config['general_listen'] = click.prompt('Which address should the relay listen on?', default=app.config.listen) app.config['general_listen'] = click.prompt(
'Which address should the relay listen on?',
default = app.config.listen
)
while True: while True:
app.config['general_port'] = click.prompt('What TCP port should the relay listen on?', default=app.config.port, type=int) app.config['general_port'] = click.prompt(
'What TCP port should the relay listen on?',
default = app.config.port,
type = int
)
break break
app.config['database_type'] = click.prompt(
'What database backend would you like to use for the relay?',
default = app.config.dbtype,
type = click.Choice(['sqlite', 'postgresql', 'mysql']),
show_choices = True
)
if app.config.dbtype == 'sqlite':
app.config['sqlite_database'] = click.prompt(
'Where would you like to store your database file? Relative paths are relative to the config file location.',
default = app.config['sqlite_database']
)
else:
dbconfig = app.config.dbconfig
app.config.hostname = click.prompt(
'What address is your database listening on?',
default = dbconfig.hostname
) or None
app.config.port = click.prompt(
'What port is your database listening on?',
default = dbconfig.port
) or None
app.config.database = click.prompt(
'What would you like the name of the database be?',
default = dbconfig.database
) or None
app.config.username = click.prompt(
'Which user will be connecting to the database?',
default = dbconfig.username
) or None
app.config.password = click.prompt(
'What is the database user\'s password?',
default = dbconfig.password
) or None
app.config.save() app.config.save()
with app.database.session as s: with app.database.session as s:

View file

@ -1,5 +1,4 @@
aiohttp>=3.8.0 aiohttp>=3.8.0
appdirs>=1.4.4
aputils@https://git.barkshark.xyz/barkshark/aputils/archive/0.1.3.tar.gz aputils@https://git.barkshark.xyz/barkshark/aputils/archive/0.1.3.tar.gz
cachetools>=5.2.0 cachetools>=5.2.0
click>=8.1.2 click>=8.1.2