diff --git a/book/changelog.md b/book/changelog.md index 5b2da825..039ed467 100644 --- a/book/changelog.md +++ b/book/changelog.md @@ -81,27 +81,26 @@ This will be the first release of Grapevine since it was forked from Conduit database backends. * The latter two commands had poor UX and didn't have any noticable effect on memory consumption. +9. **BREAKING:** Remove the `global.conduit_cache_capacity_modifier` and + `global.pdu_cache_capacity` configuration options. + ([!124](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/124)) + * Instead, it is now possible to configure each cache capacity individually. ### Changed -1. **BREAKING:** Rename `conduit_cache_capacity_modifier` configuration option - to `cache_capacity_modifier`. - ([5619d7e](https://gitlab.computer.surgery/matrix/grapevine/-/commit/5619d7e3180661731800e253b558b88b407d2ae7)) - * If you are explicitly setting this configuration option, make sure to - change its name before updating. -2. **BREAKING:** Rename Conduit to Grapevine. +1. **BREAKING:** Rename Conduit to Grapevine. ([360e020](https://gitlab.computer.surgery/matrix/grapevine/-/commit/360e020b644bd012ed438708b661a25fbd124f68)) * The `CONDUIT_VERSION_EXTRA` build-time environment variable has been renamed to `GRAPEVINE_VERSION_EXTRA`. This change only affects distribution packagers or non-Nix users who are building from source. If you fall into one of those categories *and* were explicitly setting this environment variable, make sure to change its name before building Grapevine. -3. **BREAKING:** Change the default port from 8000 to 6167. +2. **BREAKING:** Change the default port from 8000 to 6167. ([f205280](https://gitlab.computer.surgery/matrix/grapevine/-/commit/f2052805201f0685d850592b1c96f4861c58fb22)) * If you relied on the default port being 8000, either update your other configuration to use the new port, or explicitly configure Grapevine's port to 8000. -4. Improve tracing spans and events. +3. Improve tracing spans and events. ([!11 (a275db3)](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/11/diffs?commit_id=a275db3847b8d5aaa0c651a686c19cfbf9fdb8b5) (merged as [5172f66](https://gitlab.computer.surgery/matrix/grapevine/-/commit/5172f66c1a90e0e97b67be2897ae59fbc00208a4)), [!11 (a275db3)](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/11/diffs?commit_id=a275db3847b8d5aaa0c651a686c19cfbf9fdb8b5) @@ -117,31 +116,31 @@ This will be the first release of Grapevine since it was forked from Conduit [!69](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/69), [!102](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/102), [!127](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/127)) -5. Stop returning unnecessary member counts from `/_matrix/client/{r0,v3}/sync`. +4. Stop returning unnecessary member counts from `/_matrix/client/{r0,v3}/sync`. ([!12](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/12)) -6. **BREAKING:** Allow federation by default. +5. **BREAKING:** Allow federation by default. ([!24](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/24)) * If you relied on federation being disabled by default, make sure to explicitly disable it before upgrading. -7. **BREAKING:** Remove the `[global]` section from the configuration file. +6. **BREAKING:** Remove the `[global]` section from the configuration file. ([!38](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/38)) * Details on how to migrate can be found in the merge request's description. -8. **BREAKING:** Allow specifying multiple transport listeners in the +7. **BREAKING:** Allow specifying multiple transport listeners in the configuration file. ([!39](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/39)) * Details on how to migrate can be found in the merge request's description. -9. Increase default log level so that span information is included. +8. Increase default log level so that span information is included. ([!50](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/50)) -10. **BREAKING:** Reorganize config into sections. - ([!49](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/49)) - * Details on how to migrate can be found in the merge request's description. -11. Try to generate thumbnails for remote media ourselves if the federation +9. **BREAKING:** Reorganize config into sections. + ([!49](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/49)) + * Details on how to migrate can be found in the merge request's description. +10. Try to generate thumbnails for remote media ourselves if the federation thumbnail request fails. ([!58](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/58)) -12. **BREAKING:** Disable unauthenticated access to media by default, set the +11. **BREAKING:** Disable unauthenticated access to media by default, set the `serve_media_unauthenticated` config option to `true` to enable it. ([!103](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/103)) -13. **BREAKING:** Split CLI into multiple subcommands. The CLI invocation to run +12. **BREAKING:** Split CLI into multiple subcommands. The CLI invocation to run the server is now behind the `serve` command, so `grapevine --config ...` becomes `grapevine serve --config ...`. ([!108](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/108)) @@ -275,3 +274,5 @@ This will be the first release of Grapevine since it was forked from Conduit 24. Add a `check-config` CLI subcommand to check whether the configuration file is valid. ([!121](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/121)) +25. Add configuration options to tune the value of each cache individually. + ([!124](https://gitlab.computer.surgery/matrix/grapevine/-/merge_requests/124)) diff --git a/src/config.rs b/src/config.rs index 5c9ae2ee..54ef0e79 100644 --- a/src/config.rs +++ b/src/config.rs @@ -46,11 +46,8 @@ pub(crate) struct Config { pub(crate) database: DatabaseConfig, #[serde(default)] pub(crate) federation: FederationConfig, - - #[serde(default = "default_cache_capacity_modifier")] - pub(crate) cache_capacity_modifier: f64, - #[serde(default = "default_pdu_cache_capacity")] - pub(crate) pdu_cache_capacity: usize, + #[serde(default)] + pub(crate) cache: CacheConfig, #[serde(default = "default_cleanup_second_interval")] pub(crate) cleanup_second_interval: u32, #[serde(default = "default_max_request_size")] @@ -77,6 +74,38 @@ pub(crate) struct Config { pub(crate) emergency_password: Option, } +#[derive(Debug, Deserialize)] +#[serde(deny_unknown_fields, default)] +pub(crate) struct CacheConfig { + pub(crate) pdu: usize, + pub(crate) auth_chain: usize, + pub(crate) short_eventid: usize, + pub(crate) eventid_short: usize, + pub(crate) statekey_short: usize, + pub(crate) short_statekey: usize, + pub(crate) server_visibility: usize, + pub(crate) user_visibility: usize, + pub(crate) state_info: usize, + pub(crate) roomid_spacechunk: usize, +} + +impl Default for CacheConfig { + fn default() -> Self { + Self { + pdu: 150_000, + auth_chain: 100_000, + short_eventid: 100_000, + eventid_short: 100_000, + statekey_short: 100_000, + short_statekey: 100_000, + server_visibility: 100, + user_visibility: 100, + state_info: 100, + roomid_spacechunk: 200, + } + } +} + #[derive(Debug, Deserialize)] #[serde(deny_unknown_fields)] pub(crate) struct ServerDiscovery { @@ -427,19 +456,11 @@ fn default_db_cache_capacity_mb() -> f64 { 300.0 } -fn default_cache_capacity_modifier() -> f64 { - 1.0 -} - #[cfg(feature = "rocksdb")] fn default_rocksdb_max_open_files() -> i32 { 1000 } -fn default_pdu_cache_capacity() -> usize { - 150_000 -} - fn default_cleanup_second_interval() -> u32 { // every minute 60 diff --git a/src/service.rs b/src/service.rs index 56a1c8a6..85e7a302 100644 --- a/src/service.rs +++ b/src/service.rs @@ -67,14 +67,7 @@ impl Services { alias: rooms::alias::Service::new(db), auth_chain: rooms::auth_chain::Service::new( db, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100_000.0 * config.cache_capacity_modifier) as usize - }, + config.cache.auth_chain, ), directory: db, edus: rooms::edus::Service { @@ -91,89 +84,30 @@ impl Services { search: db, short: rooms::short::Service::new( db, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100_000.0 * config.cache_capacity_modifier) as usize - }, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100_000.0 * config.cache_capacity_modifier) as usize - }, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100_000.0 * config.cache_capacity_modifier) as usize - }, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100_000.0 * config.cache_capacity_modifier) as usize - }, + config.cache.short_eventid, + config.cache.eventid_short, + config.cache.statekey_short, + config.cache.short_statekey, ), state: rooms::state::Service { db, }, state_accessor: rooms::state_accessor::Service::new( db, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100.0 * config.cache_capacity_modifier) as usize - }, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100.0 * config.cache_capacity_modifier) as usize - }, + config.cache.server_visibility, + config.cache.user_visibility, ), state_cache: rooms::state_cache::Service::new(db), state_compressor: rooms::state_compressor::Service::new( db, - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (100.0 * config.cache_capacity_modifier) as usize - }, - ), - timeline: rooms::timeline::Service::new( - db, - config.pdu_cache_capacity, + config.cache.state_info, ), + timeline: rooms::timeline::Service::new(db, config.cache.pdu), threads: rooms::threads::Service { db, }, spaces: rooms::spaces::Service::new( - #[allow( - clippy::as_conversions, - clippy::cast_sign_loss, - clippy::cast_possible_truncation - )] - { - (200.0 * config.cache_capacity_modifier) as usize - }, + config.cache.roomid_spacechunk, ), user: db, }, diff --git a/tests/integrations/snapshots/integrations__check_config__invalid_keys@stderr.snap b/tests/integrations/snapshots/integrations__check_config__invalid_keys@stderr.snap index 942cce3b..2b7ce043 100644 --- a/tests/integrations/snapshots/integrations__check_config__invalid_keys@stderr.snap +++ b/tests/integrations/snapshots/integrations__check_config__invalid_keys@stderr.snap @@ -1,7 +1,6 @@ --- source: tests/integrations/check_config.rs description: A config with invalid keys fails -snapshot_kind: text --- Error: failed to validate configuration Caused by: failed to parse configuration file "tests/integrations/fixtures/check_config/invalid-keys.toml" @@ -9,4 +8,4 @@ Error: failed to validate configuration | 1 | some_name = "example.com" | ^^^^^^^^^ -unknown field `some_name`, expected one of `conduit_compat`, `listen`, `tls`, `server_name`, `server_discovery`, `database`, `federation`, `cache_capacity_modifier`, `pdu_cache_capacity`, `cleanup_second_interval`, `max_request_size`, `allow_registration`, `registration_token`, `allow_encryption`, `allow_room_creation`, `serve_media_unauthenticated`, `default_room_version`, `proxy`, `jwt_secret`, `observability`, `turn`, `emergency_password` +unknown field `some_name`, expected one of `conduit_compat`, `listen`, `tls`, `server_name`, `server_discovery`, `database`, `federation`, `cache`, `cleanup_second_interval`, `max_request_size`, `allow_registration`, `registration_token`, `allow_encryption`, `allow_room_creation`, `serve_media_unauthenticated`, `default_room_version`, `proxy`, `jwt_secret`, `observability`, `turn`, `emergency_password`