mirror of
https://gitlab.computer.surgery/matrix/grapevine.git
synced 2025-12-18 16:21:24 +01:00
8eab6eea20
One thing that might be neat in the future is noticing differing results while the tests are still running, and modifying the log messages to indicate them. I can imagine situations where you would want to abort the test run immediately after seeing the first regression.
14 KiB
14 KiB
| 1 | test | result |
|---|---|---|
| 2 | GLOBAL | FAIL |
| 3 | TestACLs | PASS |
| 4 | TestBannedUserCannotSendJoin | FAIL |
| 5 | TestCannotSendKnockViaSendKnockInMSC3787Room | FAIL |
| 6 | TestCannotSendKnockViaSendKnockInMSC3787Room/event_with_mismatched_state_key | FAIL |
| 7 | TestCannotSendKnockViaSendKnockInMSC3787Room/invite_event | FAIL |
| 8 | TestCannotSendKnockViaSendKnockInMSC3787Room/join_event | FAIL |
| 9 | TestCannotSendKnockViaSendKnockInMSC3787Room/leave_event | FAIL |
| 10 | TestCannotSendKnockViaSendKnockInMSC3787Room/non-state_membership_event | FAIL |
| 11 | TestCannotSendKnockViaSendKnockInMSC3787Room/regular_event | FAIL |
| 12 | TestCannotSendNonJoinViaSendJoinV1 | FAIL |
| 13 | TestCannotSendNonJoinViaSendJoinV1/event_with_mismatched_state_key | PASS |
| 14 | TestCannotSendNonJoinViaSendJoinV1/invite_event | PASS |
| 15 | TestCannotSendNonJoinViaSendJoinV1/knock_event | PASS |
| 16 | TestCannotSendNonJoinViaSendJoinV1/leave_event | FAIL |
| 17 | TestCannotSendNonJoinViaSendJoinV1/non-state_membership_event | PASS |
| 18 | TestCannotSendNonJoinViaSendJoinV1/regular_event | FAIL |
| 19 | TestCannotSendNonJoinViaSendJoinV2 | FAIL |
| 20 | TestCannotSendNonJoinViaSendJoinV2/event_with_mismatched_state_key | PASS |
| 21 | TestCannotSendNonJoinViaSendJoinV2/invite_event | PASS |
| 22 | TestCannotSendNonJoinViaSendJoinV2/knock_event | PASS |
| 23 | TestCannotSendNonJoinViaSendJoinV2/leave_event | FAIL |
| 24 | TestCannotSendNonJoinViaSendJoinV2/non-state_membership_event | PASS |
| 25 | TestCannotSendNonJoinViaSendJoinV2/regular_event | FAIL |
| 26 | TestCannotSendNonKnockViaSendKnock | FAIL |
| 27 | TestCannotSendNonLeaveViaSendLeaveV1 | FAIL |
| 28 | TestCannotSendNonLeaveViaSendLeaveV1/event_with_mismatched_state_key | FAIL |
| 29 | TestCannotSendNonLeaveViaSendLeaveV1/invite_event | FAIL |
| 30 | TestCannotSendNonLeaveViaSendLeaveV1/join_event | FAIL |
| 31 | TestCannotSendNonLeaveViaSendLeaveV1/knock_event | FAIL |
| 32 | TestCannotSendNonLeaveViaSendLeaveV1/non-state_membership_event | FAIL |
| 33 | TestCannotSendNonLeaveViaSendLeaveV1/regular_event | FAIL |
| 34 | TestCannotSendNonLeaveViaSendLeaveV2 | FAIL |
| 35 | TestCannotSendNonLeaveViaSendLeaveV2/event_with_mismatched_state_key | FAIL |
| 36 | TestCannotSendNonLeaveViaSendLeaveV2/invite_event | FAIL |
| 37 | TestCannotSendNonLeaveViaSendLeaveV2/join_event | FAIL |
| 38 | TestCannotSendNonLeaveViaSendLeaveV2/knock_event | FAIL |
| 39 | TestCannotSendNonLeaveViaSendLeaveV2/non-state_membership_event | FAIL |
| 40 | TestCannotSendNonLeaveViaSendLeaveV2/regular_event | FAIL |
| 41 | TestClientSpacesSummary | FAIL |
| 42 | TestClientSpacesSummary/max_depth | FAIL |
| 43 | TestClientSpacesSummary/pagination | FAIL |
| 44 | TestClientSpacesSummary/query_whole_graph | FAIL |
| 45 | TestClientSpacesSummary/redact_link | FAIL |
| 46 | TestClientSpacesSummary/suggested_only | FAIL |
| 47 | TestClientSpacesSummaryJoinRules | FAIL |
| 48 | TestContentMediaV1 | PASS |
| 49 | TestDeviceListsUpdateOverFederation | FAIL |
| 50 | TestDeviceListsUpdateOverFederation/good_connectivity | FAIL |
| 51 | TestDeviceListsUpdateOverFederation/interrupted_connectivity | FAIL |
| 52 | TestDeviceListsUpdateOverFederation/stopped_server | FAIL |
| 53 | TestDeviceListsUpdateOverFederationOnRoomJoin | FAIL |
| 54 | TestEventAuth | FAIL |
| 55 | TestFederatedClientSpaces | FAIL |
| 56 | TestFederationKeyUploadQuery | FAIL |
| 57 | TestFederationKeyUploadQuery/Can_claim_remote_one_time_key_using_POST | FAIL |
| 58 | TestFederationKeyUploadQuery/Can_query_remote_device_keys_using_POST | FAIL |
| 59 | TestFederationRedactSendsWithoutEvent | PASS |
| 60 | TestFederationRejectInvite | FAIL |
| 61 | TestFederationRoomsInvite | FAIL |
| 62 | TestFederationRoomsInvite/Parallel | FAIL |
| 63 | TestFederationRoomsInvite/Parallel/Invited_user_can_reject_invite_over_federation | FAIL |
| 64 | TestFederationRoomsInvite/Parallel/Invited_user_can_reject_invite_over_federation_for_empty_room | PASS |
| 65 | TestFederationRoomsInvite/Parallel/Invited_user_can_reject_invite_over_federation_several_times | FAIL |
| 66 | TestFederationRoomsInvite/Parallel/Invited_user_has_'is_direct'_flag_in_prev_content_after_joining | PASS |
| 67 | TestFederationRoomsInvite/Parallel/Remote_invited_user_can_see_room_metadata | PASS |
| 68 | TestFederationThumbnail | PASS |
| 69 | TestGetMissingEventsGapFilling | FAIL |
| 70 | TestInboundCanReturnMissingEvents | FAIL |
| 71 | TestInboundCanReturnMissingEvents/Inbound_federation_can_return_missing_events_for_invited_visibility | FAIL |
| 72 | TestInboundCanReturnMissingEvents/Inbound_federation_can_return_missing_events_for_joined_visibility | FAIL |
| 73 | TestInboundCanReturnMissingEvents/Inbound_federation_can_return_missing_events_for_shared_visibility | FAIL |
| 74 | TestInboundCanReturnMissingEvents/Inbound_federation_can_return_missing_events_for_world_readable_visibility | FAIL |
| 75 | TestInboundFederationKeys | PASS |
| 76 | TestInboundFederationProfile | PASS |
| 77 | TestInboundFederationProfile/Inbound_federation_can_query_profile_data | PASS |
| 78 | TestInboundFederationProfile/Non-numeric_ports_in_server_names_are_rejected | PASS |
| 79 | TestInboundFederationRejectsEventsWithRejectedAuthEvents | FAIL |
| 80 | TestIsDirectFlagFederation | PASS |
| 81 | TestIsDirectFlagLocal | PASS |
| 82 | TestJoinFederatedRoomFailOver | PASS |
| 83 | TestJoinFederatedRoomFromApplicationServiceBridgeUser | FAIL |
| 84 | TestJoinFederatedRoomFromApplicationServiceBridgeUser/join_remote_federated_room_as_application_service_user | FAIL |
| 85 | TestJoinFederatedRoomWithUnverifiableEvents | PASS |
| 86 | TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_missing_signatures_shouldn't_block_room_join | PASS |
| 87 | TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_with_bad_signatures_shouldn't_block_room_join | PASS |
| 88 | TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_with_state_with_unverifiable_auth_events_shouldn't_block_room_join | PASS |
| 89 | TestJoinFederatedRoomWithUnverifiableEvents//send_join_response_with_unobtainable_keys_shouldn't_block_room_join | PASS |
| 90 | TestJoinViaRoomIDAndServerName | PASS |
| 91 | TestJumpToDateEndpoint | FAIL |
| 92 | TestJumpToDateEndpoint/parallel | FAIL |
| 93 | TestJumpToDateEndpoint/parallel/federation | FAIL |
| 94 | TestJumpToDateEndpoint/parallel/federation/can_paginate_after_getting_remote_event_from_timestamp_to_event_endpoint | FAIL |
| 95 | TestJumpToDateEndpoint/parallel/federation/looking_backwards,_should_be_able_to_find_event_that_was_sent_before_we_joined | FAIL |
| 96 | TestJumpToDateEndpoint/parallel/federation/looking_forwards,_should_be_able_to_find_event_that_was_sent_before_we_joined | FAIL |
| 97 | TestJumpToDateEndpoint/parallel/federation/when_looking_backwards_before_the_room_was_created,_should_be_able_to_find_event_that_was_imported | FAIL |
| 98 | TestJumpToDateEndpoint/parallel/should_find_event_after_given_timestmap | FAIL |
| 99 | TestJumpToDateEndpoint/parallel/should_find_event_before_given_timestmap | FAIL |
| 100 | TestJumpToDateEndpoint/parallel/should_find_next_event_topologically_after_given_timestmap_when_all_message_timestamps_are_the_same | FAIL |
| 101 | TestJumpToDateEndpoint/parallel/should_find_next_event_topologically_before_given_timestamp_when_all_message_timestamps_are_the_same | FAIL |
| 102 | TestJumpToDateEndpoint/parallel/should_find_nothing_after_the_latest_timestmap | PASS |
| 103 | TestJumpToDateEndpoint/parallel/should_find_nothing_before_the_earliest_timestmap | PASS |
| 104 | TestJumpToDateEndpoint/parallel/should_not_be_able_to_query_a_private_room_you_are_not_a_member_of | FAIL |
| 105 | TestJumpToDateEndpoint/parallel/should_not_be_able_to_query_a_public_room_you_are_not_a_member_of | FAIL |
| 106 | TestKnockRoomsInPublicRoomsDirectory | FAIL |
| 107 | TestKnockRoomsInPublicRoomsDirectoryInMSC3787Room | FAIL |
| 108 | TestKnocking | FAIL |
| 109 | TestKnockingInMSC3787Room | FAIL |
| 110 | TestKnockingInMSC3787Room/A_user_can_knock_on_a_room_without_a_reason | FAIL |
| 111 | TestKnockingInMSC3787Room/A_user_can_knock_on_a_room_without_a_reason#01 | FAIL |
| 112 | TestKnockingInMSC3787Room/A_user_cannot_knock_on_a_room_they_are_already_in | FAIL |
| 113 | TestKnockingInMSC3787Room/A_user_cannot_knock_on_a_room_they_are_already_in#01 | FAIL |
| 114 | TestKnockingInMSC3787Room/A_user_cannot_knock_on_a_room_they_are_already_invited_to | FAIL |
| 115 | TestKnockingInMSC3787Room/A_user_cannot_knock_on_a_room_they_are_already_invited_to#01 | FAIL |
| 116 | TestKnockingInMSC3787Room/A_user_in_the_room_can_accept_a_knock | PASS |
| 117 | TestKnockingInMSC3787Room/A_user_in_the_room_can_accept_a_knock#01 | PASS |
| 118 | TestKnockingInMSC3787Room/A_user_in_the_room_can_reject_a_knock | FAIL |
| 119 | TestKnockingInMSC3787Room/A_user_in_the_room_can_reject_a_knock#01 | FAIL |
| 120 | TestKnockingInMSC3787Room/A_user_that_has_already_knocked_is_allowed_to_knock_again_on_the_same_room | FAIL |
| 121 | TestKnockingInMSC3787Room/A_user_that_has_already_knocked_is_allowed_to_knock_again_on_the_same_room#01 | FAIL |
| 122 | TestKnockingInMSC3787Room/A_user_that_has_knocked_on_a_local_room_can_rescind_their_knock_and_then_knock_again | FAIL |
| 123 | TestKnockingInMSC3787Room/A_user_that_is_banned_from_a_room_cannot_knock_on_it | FAIL |
| 124 | TestKnockingInMSC3787Room/A_user_that_is_banned_from_a_room_cannot_knock_on_it#01 | FAIL |
| 125 | TestKnockingInMSC3787Room/Attempting_to_join_a_room_with_join_rule_'knock'_without_an_invite_should_fail | FAIL |
| 126 | TestKnockingInMSC3787Room/Attempting_to_join_a_room_with_join_rule_'knock'_without_an_invite_should_fail#01 | FAIL |
| 127 | TestKnockingInMSC3787Room/Change_the_join_rule_of_a_room_from_'invite'_to_'knock' | PASS |
| 128 | TestKnockingInMSC3787Room/Change_the_join_rule_of_a_room_from_'invite'_to_'knock'#01 | PASS |
| 129 | TestKnockingInMSC3787Room/Knocking_on_a_room_with_a_join_rule_other_than_'knock'_should_fail | FAIL |
| 130 | TestKnockingInMSC3787Room/Knocking_on_a_room_with_a_join_rule_other_than_'knock'_should_fail#01 | FAIL |
| 131 | TestKnockingInMSC3787Room/Knocking_on_a_room_with_join_rule_'knock'_should_succeed | FAIL |
| 132 | TestKnockingInMSC3787Room/Knocking_on_a_room_with_join_rule_'knock'_should_succeed#01 | FAIL |
| 133 | TestKnockingInMSC3787Room/Users_in_the_room_see_a_user's_membership_update_when_they_knock | FAIL |
| 134 | TestKnockingInMSC3787Room/Users_in_the_room_see_a_user's_membership_update_when_they_knock#01 | FAIL |
| 135 | TestLocalPngThumbnail | PASS |
| 136 | TestLocalPngThumbnail/test_/_matrix/client/v1/media_endpoint | PASS |
| 137 | TestMediaFilenames | FAIL |
| 138 | TestMediaFilenames/Parallel | FAIL |
| 139 | TestMediaFilenames/Parallel/ASCII | FAIL |
| 140 | TestMediaFilenames/Parallel/ASCII/Can_download_file_'ascii' | FAIL |
| 141 | TestMediaFilenames/Parallel/ASCII/Can_download_file_'ascii'_over_/_matrix/client/v1/media/download | FAIL |
| 142 | TestMediaFilenames/Parallel/ASCII/Can_download_file_'name;with;semicolons' | FAIL |
| 143 | TestMediaFilenames/Parallel/ASCII/Can_download_file_'name;with;semicolons'_over_/_matrix/client/v1/media/download | FAIL |
| 144 | TestMediaFilenames/Parallel/ASCII/Can_download_file_'name_with_spaces' | FAIL |
| 145 | TestMediaFilenames/Parallel/ASCII/Can_download_file_'name_with_spaces'_over_/_matrix/client/v1/media/download | FAIL |
| 146 | TestMediaFilenames/Parallel/ASCII/Can_download_specifying_a_different_ASCII_file_name | PASS |
| 147 | TestMediaFilenames/Parallel/ASCII/Can_download_specifying_a_different_ASCII_file_name_over__matrix/client/v1/media/download | PASS |
| 148 | TestMediaFilenames/Parallel/ASCII/Can_upload_with_ASCII_file_name | PASS |
| 149 | TestMediaFilenames/Parallel/Unicode | FAIL |
| 150 | TestMediaFilenames/Parallel/Unicode/Can_download_specifying_a_different_Unicode_file_name | PASS |
| 151 | TestMediaFilenames/Parallel/Unicode/Can_download_specifying_a_different_Unicode_file_name_over__matrix/client/v1/media/download | PASS |
| 152 | TestMediaFilenames/Parallel/Unicode/Can_download_with_Unicode_file_name_locally | FAIL |
| 153 | TestMediaFilenames/Parallel/Unicode/Can_download_with_Unicode_file_name_locally_over__matrix/client/v1/media/download | FAIL |
| 154 | TestMediaFilenames/Parallel/Unicode/Can_download_with_Unicode_file_name_over_federation | FAIL |
| 155 | TestMediaFilenames/Parallel/Unicode/Can_download_with_Unicode_file_name_over_federation_via__matrix/client/v1/media/download | FAIL |
| 156 | TestMediaFilenames/Parallel/Unicode/Can_upload_with_Unicode_file_name | PASS |
| 157 | TestMediaFilenames/Parallel/Unicode/Will_serve_safe_media_types_as_inline | SKIP |
| 158 | TestMediaFilenames/Parallel/Unicode/Will_serve_safe_media_types_as_inline_via__matrix/client/v1/media/download | SKIP |
| 159 | TestMediaFilenames/Parallel/Unicode/Will_serve_safe_media_types_with_parameters_as_inline | SKIP |
| 160 | TestMediaFilenames/Parallel/Unicode/Will_serve_safe_media_types_with_parameters_as_inline_via__matrix/client/v1/media/download | SKIP |
| 161 | TestMediaFilenames/Parallel/Unicode/Will_serve_unsafe_media_types_as_attachments | SKIP |
| 162 | TestMediaFilenames/Parallel/Unicode/Will_serve_unsafe_media_types_as_attachments_via__matrix/client/v1/media/download | SKIP |
| 163 | TestMediaWithoutFileName | FAIL |
| 164 | TestMediaWithoutFileName/parallel | FAIL |
| 165 | TestMediaWithoutFileName/parallel/Can_download_without_a_file_name_locally | PASS |
| 166 | TestMediaWithoutFileName/parallel/Can_download_without_a_file_name_over_federation | FAIL |
| 167 | TestMediaWithoutFileName/parallel/Can_upload_without_a_file_name | PASS |
| 168 | TestMediaWithoutFileNameCSMediaV1 | PASS |
| 169 | TestMediaWithoutFileNameCSMediaV1/parallel | PASS |
| 170 | TestMediaWithoutFileNameCSMediaV1/parallel/Can_download_without_a_file_name_locally | PASS |
| 171 | TestMediaWithoutFileNameCSMediaV1/parallel/Can_download_without_a_file_name_over_federation | PASS |
| 172 | TestMediaWithoutFileNameCSMediaV1/parallel/Can_upload_without_a_file_name | PASS |
| 173 | TestNetworkPartitionOrdering | FAIL |
| 174 | TestOutboundFederationIgnoresMissingEventWithBadJSONForRoomVersion6 | FAIL |
| 175 | TestOutboundFederationProfile | PASS |
| 176 | TestOutboundFederationProfile/Outbound_federation_can_query_profile_data | PASS |
| 177 | TestOutboundFederationSend | PASS |
| 178 | TestRemoteAliasRequestsUnderstandUnicode | PASS |
| 179 | TestRemotePngThumbnail | PASS |
| 180 | TestRemotePngThumbnail/test_/_matrix/client/v1/media_endpoint | PASS |
| 181 | TestRemotePresence | FAIL |
| 182 | TestRemotePresence/Presence_changes_are_also_reported_to_remote_room_members | FAIL |
| 183 | TestRemotePresence/Presence_changes_to_UNAVAILABLE_are_reported_to_remote_room_members | FAIL |
| 184 | TestRemoteTyping | FAIL |
| 185 | TestRestrictedRoomsLocalJoin | FAIL |
| 186 | TestRestrictedRoomsLocalJoinInMSC3787Room | FAIL |
| 187 | TestRestrictedRoomsLocalJoinInMSC3787Room/Join_should_fail_initially | PASS |
| 188 | TestRestrictedRoomsLocalJoinInMSC3787Room/Join_should_fail_when_left_allowed_room | FAIL |
| 189 | TestRestrictedRoomsLocalJoinInMSC3787Room/Join_should_fail_with_mangled_join_rules | PASS |
| 190 | TestRestrictedRoomsLocalJoinInMSC3787Room/Join_should_succeed_when_invited | FAIL |
| 191 | TestRestrictedRoomsLocalJoinInMSC3787Room/Join_should_succeed_when_joined_to_allowed_room | PASS |
| 192 | TestRestrictedRoomsRemoteJoin | FAIL |
| 193 | TestRestrictedRoomsRemoteJoinFailOver | FAIL |
| 194 | TestRestrictedRoomsRemoteJoinFailOverInMSC3787Room | FAIL |
| 195 | TestRestrictedRoomsRemoteJoinInMSC3787Room | FAIL |
| 196 | TestRestrictedRoomsRemoteJoinInMSC3787Room/Join_should_fail_initially | PASS |
| 197 | TestRestrictedRoomsRemoteJoinInMSC3787Room/Join_should_fail_when_left_allowed_room | PASS |
| 198 | TestRestrictedRoomsRemoteJoinInMSC3787Room/Join_should_fail_with_mangled_join_rules | PASS |
| 199 | TestRestrictedRoomsRemoteJoinInMSC3787Room/Join_should_succeed_when_invited | PASS |
| 200 | TestRestrictedRoomsRemoteJoinInMSC3787Room/Join_should_succeed_when_joined_to_allowed_room | FAIL |
| 201 | TestRestrictedRoomsRemoteJoinLocalUser | FAIL |
| 202 | TestRestrictedRoomsRemoteJoinLocalUserInMSC3787Room | FAIL |
| 203 | TestRestrictedRoomsSpacesSummaryFederation | FAIL |
| 204 | TestRestrictedRoomsSpacesSummaryLocal | FAIL |
| 205 | TestSendJoinPartialStateResponse | SKIP |
| 206 | TestSyncOmitsStateChangeOnFilteredEvents | PASS |
| 207 | TestToDeviceMessagesOverFederation | FAIL |
| 208 | TestToDeviceMessagesOverFederation/good_connectivity | PASS |
| 209 | TestToDeviceMessagesOverFederation/interrupted_connectivity | FAIL |
| 210 | TestToDeviceMessagesOverFederation/stopped_server | FAIL |
| 211 | TestUnbanViaInvite | FAIL |
| 212 | TestUnknownEndpoints | FAIL |
| 213 | TestUnknownEndpoints/Client-server_endpoints | PASS |
| 214 | TestUnknownEndpoints/Key_endpoints | FAIL |
| 215 | TestUnknownEndpoints/Media_endpoints | PASS |
| 216 | TestUnknownEndpoints/Server-server_endpoints | PASS |
| 217 | TestUnknownEndpoints/Unknown_prefix | PASS |
| 218 | TestUnrejectRejectedEvents | FAIL |
| 219 | TestUserAppearsInChangedDeviceListOnJoinOverFederation | PASS |
| 220 | TestWriteMDirectAccountData | PASS |