Commit graph

2640 commits

Author SHA1 Message Date
Benjamin Lee
5315bac0c5
split out separate error type for serve command 2024-09-21 14:11:40 -07:00
Benjamin Lee
86515d53cc
move 'serve' command logic into a submodule of 'cli'
The changes to 'main.rs' and 'cli/serve.rs' in this commit are almost
pure code-motion.
2024-09-21 14:11:39 -07:00
Benjamin Lee
be87774a3b
set up structure for multiple cli commands
The previous cli is now behind the 'serve' subcommand.
2024-09-21 14:11:26 -07:00
Charles Hall
1ee3bbb316
oops, i dropped my fork
The maintainers had a discussion internally and decided it's unlikely
that we'll have the capacity to try to do a rewrite, which was the
original reason for the suffix's presence. So, now can get rid of it.
2024-09-20 16:52:05 -07:00
Charles Hall
d388994657
rewrite media key parser
Fixes a regression in e2cba15ed2 where the
Content-Type and Content-Disposition parts are extracted in the wrong
order.

Fixes a long-standing issue in b6d721374f
where the Content-Type part was allowed to be completely missing rather
than present and 0 bytes long.

Improves the error messages for various parsing failures to be unique
and more obvious.
2024-09-19 15:27:10 -07:00
Charles Hall
88b009a8d4
update changelog 2024-09-19 15:23:59 -07:00
Charles Hall
b34d78a030
skip over broken keys instead of aborting
Errors will show up in the logs in this case with detailed information
about what broke.

In the future we should add some kind of database integrity check
functionality and also functionality to repair/delete broken data, but
for now this at least makes it work 99.99% of the time.
2024-09-19 15:23:59 -07:00
Charles Hall
cb3e0c620a
improve media key decoding logs
On my HS I observed 5 instances of keys with the following format:

* MXC bytes.
* A 0xFF byte.
* 4 bytes where the width and height are supposed to be, which are
  supposed to be 8 bytes in length.
* 3 consecutive 0xFF bytes. This means that the `content-type` and
  `content-disposition` sections both parse as the empty string, and
  there's an extra separator at the end too.
* Extra bytes, all of which were `image/png`.

The 4 bytes where the width and height are supposed to be were one of:

* 003ED000
* 003EE000
* 003EF001

Which seems to have some kind of pattern to it...

After much digging, we have absolutely no idea what could've caused
this. Cursed.
2024-09-19 15:23:20 -07:00
Charles Hall
d848e787d3
ignore files that were probably never created
File data is inserted into the database before being created on disk,
which means that it's possible for data to exist in the database that
doesn't exist on disk. In this case, the media deletion functions should
simply ignore this error.
2024-09-19 12:29:51 -07:00
Lambda
ca6bc74074 Fix X-Matrix signature validation for incoming requests
For HTTP/1 requests, an inbound Request's URI contains only the path and
query parameters, since there's no way to synthesize the authority part.
This is exactly what we need for the X-Matrix "uri" field.

HTTP/2 requests however can contain the :authority pseudo-header, which
is used to populate the Request's URI. Using a URL that includes an
authority breaks the signature check.

Largely inspired by conduit MR !631
(https://gitlab.com/famedly/conduit/-/merge_requests/631).

Co-authored-by: strawberry <strawberry@puppygock.gay>
2024-09-19 16:25:23 +00:00
Lambda
0d6a7eb968 Disable unauthenticated media access 2024-09-18 20:33:28 +00:00
Charles Hall
b9ee898920
require client base_url, rename from authority
The previous code used `server_name` as a fallback but in reality there
is no real relationship between `server_name` and the location clients
are supposed to make requests to.

Additionally, the `insecure` option is gone, because we now allow users
to control the entire URL, so they're free to choose the scheme.
2024-09-18 13:03:49 -07:00
Benjamin Lee
48850605b0
changelog entry for media deletion admin commands 2024-09-17 19:31:54 -07:00
Benjamin Lee
ba7b224c38
add dry-run mode to delete-remote-media-files admin command 2024-09-17 19:31:54 -07:00
Benjamin Lee
9d14c5d461
add admin command to delete all remote media files 2024-09-17 19:31:51 -07:00
Benjamin Lee
d7087c66bb
add admin command to delete individual media files 2024-09-17 19:13:54 -07:00
Benjamin Lee
7672cc8473
use OwnedMxcUri in media service
Not using `MxcData` because it borrows it's fields, and so we wouldn't
be able to return an owned `MxcData` from functions that read the db.
2024-09-15 00:32:17 -07:00
Benjamin Lee
e2cba15ed2
factor out helper for parsing media keys
Leaving this private in `database::key_value::media` because the way
the metadata is encoded in media keys is a mess. I want to fix that in
the future, and want to limit the number of things that rely on it for
now.
2024-09-15 00:32:17 -07:00
Lambda
3bb4a25c1d Include old verify keys in _matrix/key/v2/server response 2024-09-13 17:02:30 +00:00
Lambda
296824fef4 Always use local keypair instead of trying to find our own keys in cache 2024-09-13 17:02:30 +00:00
Lambda
458a7458dc Support specifying old_verify_keys in config 2024-09-13 17:02:30 +00:00
Lambda
5691cf0868 Better debugging for signing key fetching 2024-09-13 13:31:04 +00:00
Charles Hall
9e6a5e6604
update changelog 2024-09-08 14:08:32 -07:00
Charles Hall
449c27642c
hide sliding sync behind explicit option
We want to make sure users know this sliding sync impl is pretty buggy
before they attempt to use it.
2024-09-08 14:08:32 -07:00
Charles Hall
806cc0cb28
serve well-known client and server config
This way users can have a simpler time configuring this stuff and we can
worry about the spec compliance parts and specifying the same thing over
and over parts.
2024-09-08 13:35:38 -07:00
Lambda
3a55684623
Update MSRV to 1.81.0
Plus a "__CARGO_FIX_YOLO=1 cargo clippy --fix"
2024-09-08 09:54:08 -07:00
Lambda
74589043f7
Fix weird type gymnastics 2024-09-08 09:54:04 -07:00
Lambda
5c4062742f
Log curl command line for all requests at trace 2024-09-08 09:10:40 -07:00
Lambda
b0f33207fe Add wrapper types for short IDs 2024-09-06 17:20:14 +00:00
Lambda
f1642c92d1 Take iterator in calculate_hash()
Avoids unnecessary allocations.
2024-09-06 17:20:14 +00:00
Lambda
cce83beedb Properly type stored EDUs 2024-09-06 17:20:14 +00:00
Lambda
26322d5a95 Add PduId wrapper struct
Death to Vec<u8>
2024-09-06 17:20:14 +00:00
Lambda
341f4213d0 Use self instead of going through services() 2024-09-05 19:42:37 +00:00
Lambda
f52cf53931 Remove obsolete clippy #[allow]s 2024-09-05 19:42:37 +00:00
Lambda
22ce624a81 event_handler: remove AsyncRecursiveType alias, simplify signatures 2024-09-05 19:42:37 +00:00
Lambda
542e097cdf server_server: use non-deprecated create_join_event types
Instead of working with v1 types and converting them to v2 as required,
do it the other way around. Allows us to remove a very broad
#[allow(deprecated)].
2024-09-05 19:42:37 +00:00
Benjamin Lee
be14f5bddc
fetch signing keys on join even when no cached keys for origin
Silly mistake
2024-09-01 12:22:10 -07:00
Lambda
b4fecbc517 client/media: work around matrix-media-repo bug
It rejects Authenticated Media requests if X-Matrix values aren't
unnecessarily quoted:
https://github.com/t2bot/matrix-media-repo/issues/609
2024-09-01 09:42:45 +00:00
Lambda
3fe0110649 media: convert allow_remote to enum 2024-09-01 09:42:45 +00:00
Lambda
a4b7df1b3a media: use FileMeta instead of tuples 2024-09-01 09:42:45 +00:00
Lambda
1ccb1e572b media: add MediaFileKey wrapper
One more win in the fight against the Vec<u8>s
2024-09-01 09:42:45 +00:00
Lambda
84850a163d Factor content out of FileMeta
That's not what *meta*data means
2024-09-01 09:42:45 +00:00
Lambda
a06c8db996 Fetch and thumbnail original media if fetching thumbnail fails
E.g. because the remote server's thumbnail endpoint is broken in
hilarious ways: https://github.com/element-hq/synapse/issues/17518
2024-09-01 09:42:45 +00:00
Lambda
82aacdc153 Update changelog for Authenticated Media 2024-09-01 09:42:45 +00:00
Lambda
d3b6718812 server_server: implement authenticated media endpoints 2024-09-01 09:42:45 +00:00
Lambda
79053ad052 client_server: use and provide authenticated media API 2024-09-01 09:42:45 +00:00
Lambda
7f6ab63752 client_server: factor out width/height conversion 2024-09-01 09:42:45 +00:00
Lambda
edfaa83405 server_server: make outbound requests using spec v1.11
This is required for authenticated media requests, which are otherwise
performed using the unstable endpoint.
2024-09-01 09:42:45 +00:00
Lambda
94204415ee Add MxcData helper 2024-09-01 09:42:45 +00:00
Lambda
64b3c357dd media: put old API behind legacy_media import 2024-09-01 09:42:45 +00:00