Merging develop.

Fix Invalid SemVer version generation

See merge request pleroma/pleroma!1517

CHANGELOG.md

@@ -4,13 +4,20 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ## [Unreleased]
+### Security
+- OStatus: eliminate the possibility of a protocol downgrade attack.
+- OStatus: prevent following locked accounts, bypassing the approval process.
+
 ### Changed
 - **Breaking:** Configuration: A setting to explicitly disable the mailer was added, defaulting to true, if you are using a mailer add `config :pleroma, Pleroma.Emails.Mailer, enabled: true` to your config
+- **Breaking:** Configuration: `/media/` is now removed when `base_url` is configured, append `/media/` to your `base_url` config to keep the old behaviour if desired
 - Configuration: OpenGraph and TwitterCard providers enabled by default
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
 - Federation: Return 403 errors when trying to request pages from a user's follower/following collections if they have `hide_followers`/`hide_follows` set
 - NodeInfo: Return `skipThreadContainment` in `metadata` for the `skip_thread_containment` option
+- NodeInfo: Return `mailerEnabled` in `metadata`
 - Mastodon API: Unsubscribe followers when they unfollow a user
+- AdminAPI: Add "godmode" while fetching user statuses (i.e. admin can see private statuses)
 
 ### Fixed
 - Not being able to pin unlisted posts
@@ -19,12 +26,23 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Handling of search timeouts (`/api/v1/search` and `/api/v2/search`)
 - Mastodon API: Embedded relationships not being properly rendered in the Account entity of Status entity
 - Mastodon API: Add `account_id`, `type`, `offset`, and `limit` to search API (`/api/v1/search` and `/api/v2/search`)
+- Mastodon API, streaming: Fix filtering of notifications based on blocks/mutes/thread mutes
 - ActivityPub C2S: follower/following collection pages being inaccessible even when authentifucated if `hide_followers`/ `hide_follows` was set
+- Existing user id not being preserved on insert conflict
+- Rich Media: Parser failing when no TTL can be found by image TTL setters
+- Rich Media: The crawled URL is now spliced into the rich media data.
+- ActivityPub S2S: sharedInbox usage has been mostly aligned with the rules in the AP specification.
+- ActivityPub S2S: remote user deletions now work the same as local user deletions.
+- Not being able to access the Mastodon FE login page on private instances
+- Invalid SemVer version generation, when the current branch does not have commits ahead of tag/checked out on a tag
 
 ### Added
 - MRF: Support for priming the mediaproxy cache (`Pleroma.Web.ActivityPub.MRF.MediaProxyWarmingPolicy`)
 - MRF: Support for excluding specific domains from Transparency.
 - MRF: Support for filtering posts based on who they mention (`Pleroma.Web.ActivityPub.MRF.MentionPolicy`)
+- MRF (Simple Policy): Support for wildcard domains.
+- Support for wildcard domains in user domain blocks setting.
+- Configuration: `quarantined_instances` support wildcard domains.
 - Configuration: `federation_incoming_replies_max_depth` option
 - Mastodon API: Support for the [`tagged` filter](https://github.com/tootsuite/mastodon/pull/9755) in [`GET /api/v1/accounts/:id/statuses`](https://docs.joinmastodon.org/api/rest/accounts/#get-api-v1-accounts-id-statuses)
 - Mastodon API, streaming: Add support for passing the token in the `Sec-WebSocket-Protocol` header
@@ -32,12 +50,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Mastodon API: Add support for categories for custom emojis by reusing the group feature. <https://github.com/tootsuite/mastodon/pull/11196>
 - Mastodon API: Add support for muting/unmuting notifications
 - Mastodon API: Add support for the `blocked_by` attribute in the relationship API (`GET /api/v1/accounts/relationships`). <https://github.com/tootsuite/mastodon/pull/10373>
+- Mastodon API: Add support for the `domain_blocking` attribute in the relationship API (`GET /api/v1/accounts/relationships`).
 - Mastodon API: Add `pleroma.deactivated` to the Account entity
 - Mastodon API: added `/auth/password` endpoint for password reset with rate limit.
+- Mastodon API: /api/v1/accounts/:id/statuses now supports nicknames or user id
 - Admin API: Return users' tags when querying reports
 - Admin API: Return avatar and display name when querying users
 - Admin API: Allow querying user by ID
 - Admin API: Added support for `tuples`.
+- Admin API: Added endpoints to run mix tasks pleroma.config migrate_to_db & pleroma.config migrate_from_db
 - Added synchronization of following/followers counters for external users
 - Configuration: `enabled` option for `Pleroma.Emails.Mailer`, defaulting to `false`.
 - Configuration: Pleroma.Plugs.RateLimiter `bucket_name`, `params` options.
@@ -45,11 +66,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 - Twitter API: added rate limit for `/api/account/password_reset` endpoint.
 - ActivityPub: Add an internal service actor for fetching ActivityPub objects.
 - ActivityPub: Optional signing of ActivityPub object fetches.
+- Admin API: Endpoint for fetching latest user's statuses
+- Pleroma API: Add `/api/v1/pleroma/accounts/confirmation_resend?email=<email>` for resending account confirmation.
 
 ### Changed
 - Configuration: Filter.AnonymizeFilename added ability to retain file extension with custom text
 - Admin API: changed json structure for saving config settings.
 - RichMedia: parsers and their order are configured in `rich_media` config.
+- RichMedia: add the rich media ttl based on image expiration time.
 
 ## [1.0.1] - 2019-07-14
 ### Security

config/config.exs

@@ -345,7 +345,8 @@ config :pleroma, :rich_media,
     Pleroma.Web.RichMedia.Parsers.TwitterCard,
     Pleroma.Web.RichMedia.Parsers.OGP,
     Pleroma.Web.RichMedia.Parsers.OEmbed
-  ]
+  ],
+  ttl_setters: [Pleroma.Web.RichMedia.Parser.TTL.AwsSignedUrl]
 
 config :pleroma, :media_proxy,
   enabled: false,
@@ -533,7 +534,8 @@ config :pleroma, :rate_limit,
   relation_id_action: {60_000, 2},
   statuses_actions: {10_000, 15},
   status_id_action: {60_000, 3},
-  password_reset: {1_800_000, 5}
+  password_reset: {1_800_000, 5},
+  account_confirmation_resend: {8_640_000, 5}
 
 # Import environment specific config. This must remain at the bottom
 # of this file so it overrides the configuration defined above.

docs/api/admin_api.md

@@ -187,6 +187,19 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
   - On failure: `Not found`
   - On success: JSON of the user
 
+## `/api/pleroma/admin/users/:nickname_or_id/statuses`
+
+### Retrive user's latest statuses
+
+- Method: `GET`
+- Params:
+  - `nickname` or `id`
+  - *optional* `page_size`: number of statuses to return (default is `20`)
+  - *optional* `godmode`: `true`/`false` – allows to see private statuses
+- Response:
+  - On failure: `Not found`
+  - On success: JSON array of user's latest statuses
+
 ## `/api/pleroma/admin/relay`
 
 ### Follow a Relay
@@ -562,8 +575,32 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
     - 404 Not Found `"Not found"`
   - On success: 200 OK `{}`
 
+
+## `/api/pleroma/admin/config/migrate_to_db`
+### Run mix task pleroma.config migrate_to_db
+Copy settings on key `:pleroma` to DB.
+- Method `GET`
+- Params: none
+- Response:
+
+```json
+{}
+```
+
+## `/api/pleroma/admin/config/migrate_from_db`
+### Run mix task pleroma.config migrate_from_db
+Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with deletion from DB.
+- Method `GET`
+- Params: none
+- Response:
+
+```json
+{}
+```
+
 ## `/api/pleroma/admin/config`
 ### List config settings
+List config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`.
 - Method `GET`
 - Params: none
 - Response:
@@ -582,6 +619,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret
 
 ## `/api/pleroma/admin/config`
 ### Update config settings
+Updating config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`.
 Module name can be passed as string, which starts with `Pleroma`, e.g. `"Pleroma.Upload"`.
 Atom keys and values can be passed with `:` in the beginning, e.g. `":upload"`.
 Tuples can be passed as `{"tuple": ["first_val", Pleroma.Module, []]}`.

docs/api/differences_in_mastoapi_responses.md

@@ -34,7 +34,10 @@ Has these additional fields under the `pleroma` object:
 
 ## Accounts
 
-- `/api/v1/accounts/:id`: The `id` parameter can also be the `nickname` of the user. This only works in this endpoint, not the deeper nested ones for following etc.
+The `id` parameter can also be the `nickname` of the user. This only works in these endpoints, not the deeper nested ones for following etc.
+
+- `/api/v1/accounts/:id`
+- `/api/v1/accounts/:id/statuses`
 
 Has these additional fields under the `pleroma` object:
 

docs/api/pleroma_api.md

@@ -245,6 +245,14 @@ See [Admin-API](Admin-API.md)
 - PATCH `/api/v1/pleroma/accounts/update_banner`: Set/clear user banner image
 - PATCH `/api/v1/pleroma/accounts/update_background`: Set/clear user background image
 
+## `/api/v1/pleroma/accounts/confirmation_resend`
+### Resend confirmation email
+* Method `POST`
+* Params:
+    * `email`: email of that needs to be verified
+* Authentication: not required
+* Response: 204 No Content 
+
 ## `/api/v1/pleroma/mascot`
 ### Gets user mascot image
 * Method `GET`

docs/clients.md

@@ -32,10 +32,10 @@ Feel free to contact us to be added to this list!
 
 ### Fedilab
 - Homepage: <https://fedilab.app/>
-- Source Code: <https://gitlab.com/tom79/mastalab/>
-- Contact: [@tom79@mastodon.social](https://mastodon.social/users/tom79)
+- Source Code: <https://framagit.org/tom79/fedilab/>
+- Contact: [@fedilab@framapiaf.org](https://framapiaf.org/users/fedilab)
 - Platforms: Android
-- Features: Streaming Ready
+- Features: Streaming Ready, Moderation, Text Formatting 
 
 ### Nekonium
 - Homepage: [F-Droid Repository](https://repo.gdgd.jp.net/), [Google Play](https://play.google.com/store/apps/details?id=com.apps.nekonium), [Amazon](https://www.amazon.co.jp/dp/B076FXPRBC/)

docs/config/howto_set_richmedia_cache_ttl_based_on_image.md

@@ -0,0 +1,33 @@
+# How to set rich media cache ttl based on image ttl
+## Explanation
+
+Richmedia are cached without the ttl but the rich media may have image which can expire, like aws signed url.
+In such cases the old image url (expired) is returned from the media cache.
+
+So to avoid such situation we can define a module that will set ttl based on image.
+The module must adopt behaviour `Pleroma.Web.RichMedia.Parser.TTL`
+
+### Example
+
+```exs
+defmodule MyModule do
+  @behaviour Pleroma.Web.RichMedia.Parser.TTL
+
+  @impl Pleroma.Web.RichMedia.Parser.TTL
+  def ttl(data, url) do
+    image_url = Map.get(data, :image)
+    # do some parsing in the url and get the ttl of the image
+    # return ttl is unix time
+    parse_ttl_from_url(image_url)
+  end
+end
+```
+
+And update the config
+
+```exs
+config :pleroma, :rich_media,
+  ttl_setters: [Pleroma.Web.RichMedia.Parser.TTL.AwsSignedUrl, MyModule]
+```
+
+> For reference there is a parser for AWS signed URL `Pleroma.Web.RichMedia.Parser.TTL.AwsSignedUrl`, it's enabled by default.

docs/config/small_customizations.md

@@ -1,35 +1,12 @@
 # Small customizations
-Replace `dev.secret.exs` with `prod.secret.exs` according to your setup.
 
-# Thumbnail
+See also static_dir.md for visual settings.
 
-Replace `priv/static/instance/thumbnail.jpeg` with your selfie or other neat picture. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html).
-
-# Instance-specific panel
-
-![instance-specific panel demo](/uploads/296b19ec806b130e0b49b16bfe29ce8a/image.png)
-
-To show the instance specific panel, set `show_instance_panel` to `true` in `config/dev.secret.exs`. You can modify its content by editing `priv/static/instance/panel.html`.
-
-# Background
-
-You can change the background of your Pleroma instance by uploading it to `priv/static/static`, and then changing `"background"` in `config/dev.secret.exs` accordingly.
-
-# Logo
-
-![logo modification demo](/uploads/c70b14de60fa74245e7f0dcfa695ebff/image.png)
-
-If you want to give a brand to your instance, look no further. You can change the logo of your instance by uploading it to `priv/static/static`, and then changing `logo` in `config/dev.secret.exs` accordingly.
-
-# Theme
+## Theme
 
 All users of your instance will be able to change the theme they use by going to the settings (the cog in the top-right hand corner). However, if you wish to change the default theme, you can do so by editing `theme` in `config/dev.secret.exs` accordingly.
 
-# Terms of Service
-
-Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by changing `priv/static/static/terms-of-service.html`.
-
-# Message Visibility
+## Message Visibility
 
 To enable message visibility options when posting like in the Mastodon frontend, set
 `scope_options_enabled` to `true` in `config/dev.secret.exs`.

docs/config/static_dir.md

@@ -7,7 +7,13 @@ config :pleroma, :instance,
   static_dir: "instance/static/",
 ```
 
-You can overwrite this value in your configuration to use a different static instance directory.
+For example, edit `instance/static/instance/panel.html` .
+
+Alternatively, you can overwrite this value in your configuration to use a different static instance directory.
+
+This document is written assuming `instance/static/`.
+
+Or, if you want to manage your custom file in git repository, basically remove the `instance/` entry from `.gitignore`.
 
 ## robots.txt
 
@@ -18,3 +24,46 @@ If you want to generate a restrictive `robots.txt`, you can run the following mi
 ```
 mix pleroma.robots_txt disallow_all
 ```
+
+## Thumbnail
+
+Put on `instance/static/instance/thumbnail.jpeg` with your selfie or other neat picture. It will appear in [Pleroma Instances](http://distsn.org/pleroma-instances.html).
+
+## Instance-specific panel
+
+![instance-specific panel demo](/uploads/296b19ec806b130e0b49b16bfe29ce8a/image.png)
+
+Create and Edit your file on `instance/static/instance/panel.html`.
+
+## Background
+
+You can change the background of your Pleroma instance by uploading it to `instance/static/`, and then changing `background` in `config/prod.secret.exs` accordingly.
+
+If you put `instance/static/images/background.jpg`
+
+```
+config :pleroma, :frontend_configurations,
+  pleroma_fe: %{
+    background: "/images/background.jpg"
+  }
+```
+
+## Logo
+
+![logo modification demo](/uploads/c70b14de60fa74245e7f0dcfa695ebff/image.png)
+
+If you want to give a brand to your instance, You can change the logo of your instance by uploading it to `instance/static/`.
+
+Alternatively, you can specify the path with config.
+If you put `instance/static/static/mylogo-file.png`
+
+```
+config :pleroma, :frontend_configurations,
+  pleroma_fe: %{
+   logo: "/static/mylogo-file.png"
+  }
+```
+
+## Terms of Service
+
+Terms of Service will be shown to all users on the registration page. It's the best place where to write down the rules for your instance. You can modify the rules by changing `instance/static/static/terms-of-service.html`.

lib/mix/tasks/pleroma/config.ex

@@ -15,7 +15,7 @@ defmodule Mix.Tasks.Pleroma.Config do
 
       mix pleroma.config migrate_to_db
 
-  ## Transfers config from DB to file.
+  ## Transfers config from DB to file `config/env.exported_from_db.secret.exs`
 
       mix pleroma.config migrate_from_db ENV
   """

lib/mix/tasks/pleroma/database.ex

@@ -8,6 +8,7 @@ defmodule Mix.Tasks.Pleroma.Database do
   alias Pleroma.Repo
   alias Pleroma.User
   require Logger
+  require Pleroma.Constants
   import Mix.Pleroma
   use Mix.Task
 
@@ -99,10 +100,15 @@ defmodule Mix.Tasks.Pleroma.Database do
       NaiveDateTime.utc_now()
       |> NaiveDateTime.add(-(deadline * 86_400))
 
-    public = "https://www.w3.org/ns/activitystreams#Public"
-
     from(o in Object,
-      where: fragment("?->'to' \\? ? OR ?->'cc' \\? ?", o.data, ^public, o.data, ^public),
+      where:
+        fragment(
+          "?->'to' \\? ? OR ?->'cc' \\? ?",
+          o.data,
+          ^Pleroma.Constants.as_public(),
+          o.data,
+          ^Pleroma.Constants.as_public()
+        ),
       where: o.inserted_at < ^time_deadline,
       where:
         fragment("split_part(?->>'actor', '/', 3) != ?", o.data, ^Pleroma.Web.Endpoint.host())

lib/mix/tasks/pleroma/user.ex

@@ -62,6 +62,10 @@ defmodule Mix.Tasks.Pleroma.User do
 
       mix pleroma.user unsubscribe NICKNAME
 
+  ## Unsubscribe local users from an entire instance and deactivate all accounts
+
+      mix pleroma.user unsubscribe_all_from_instance INSTANCE
+
   ## Create a password reset link.
 
       mix pleroma.user reset_password NICKNAME
@@ -246,6 +250,20 @@ defmodule Mix.Tasks.Pleroma.User do
     end
   end
 
+  def run(["unsubscribe_all_from_instance", instance]) do
+    start_pleroma()
+
+    Pleroma.User.Query.build(%{nickname: "@#{instance}"})
+    |> Pleroma.RepoStreamer.chunk_stream(500)
+    |> Stream.each(fn users ->
+      users
+      |> Enum.each(fn user ->
+        run(["unsubscribe", user.nickname])
+      end)
+    end)
+    |> Stream.run()
+  end
+
   def run(["set", nickname | rest]) do
     start_pleroma()
 

lib/pleroma/activity/search.ex

@@ -9,6 +9,8 @@ defmodule Pleroma.Activity.Search do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.Visibility
 
+  require Pleroma.Constants
+
   import Ecto.Query
 
   def search(user, search_query, options \\ []) do
@@ -39,7 +41,7 @@ defmodule Pleroma.Activity.Search do
   defp restrict_public(q) do
     from([a, o] in q,
       where: fragment("?->>'type' = 'Create'", a.data),
-      where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients
+      where: ^Pleroma.Constants.as_public() in a.recipients
     )
   end
 

lib/pleroma/constants.ex

@@ -0,0 +1,9 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Constants do
+  use Const
+
+  const(as_public, do: "https://www.w3.org/ns/activitystreams#Public")
+end

lib/pleroma/flake_id.ex

@@ -66,6 +66,16 @@ defmodule Pleroma.FlakeId do
   @spec get :: binary
   def get, do: to_string(:gen_server.call(:flake, :get))
 
+  # checks that ID is is valid FlakeID
+  #
+  @spec is_flake_id?(String.t()) :: boolean
+  def is_flake_id?(id), do: is_flake_id?(String.to_charlist(id), true)
+  defp is_flake_id?([c | cs], true) when c >= ?0 and c <= ?9, do: is_flake_id?(cs, true)
+  defp is_flake_id?([c | cs], true) when c >= ?A and c <= ?Z, do: is_flake_id?(cs, true)
+  defp is_flake_id?([c | cs], true) when c >= ?a and c <= ?z, do: is_flake_id?(cs, true)
+  defp is_flake_id?([], true), do: true
+  defp is_flake_id?(_, _), do: false
+
   # -- Ecto.Type API
   @impl Ecto.Type
   def type, do: :uuid

lib/pleroma/http/connection.ex

@@ -11,6 +11,7 @@ defmodule Pleroma.HTTP.Connection do
     connect_timeout: 10_000,
     recv_timeout: 20_000,
     follow_redirect: true,
+    force_redirect: true,
     pool: :federation
   ]
   @adapter Application.get_env(:tesla, :adapter)

lib/pleroma/plugs/http_signature.ex

@@ -3,7 +3,6 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.Plugs.HTTPSignaturePlug do
-  alias Pleroma.Web.ActivityPub.Utils
   import Plug.Conn
   require Logger
 
@@ -16,38 +15,30 @@ defmodule Pleroma.Web.Plugs.HTTPSignaturePlug do
   end
 
   def call(conn, _opts) do
-    user = Utils.get_ap_id(conn.params["actor"])
-    Logger.debug("Checking sig for #{user}")
     [signature | _] = get_req_header(conn, "signature")
 
-    cond do
-      signature && String.contains?(signature, user) ->
-        # set (request-target) header to the appropriate value
-        # we also replace the digest header with the one we computed
-        conn =
-          conn
-          |> put_req_header(
-            "(request-target)",
-            String.downcase("#{conn.method}") <> " #{conn.request_path}"
-          )
-
-        conn =
-          if conn.assigns[:digest] do
-            conn
-            |> put_req_header("digest", conn.assigns[:digest])
-          else
-            conn
-          end
-
-        assign(conn, :valid_signature, HTTPSignatures.validate_conn(conn))
-
-      signature ->
-        Logger.debug("Signature not from actor")
-        assign(conn, :valid_signature, false)
-
-      true ->
-        Logger.debug("No signature header!")
+    if signature do
+      # set (request-target) header to the appropriate value
+      # we also replace the digest header with the one we computed
+      conn =
         conn
+        |> put_req_header(
+          "(request-target)",
+          String.downcase("#{conn.method}") <> " #{conn.request_path}"
+        )
+
+      conn =
+        if conn.assigns[:digest] do
+          conn
+          |> put_req_header("digest", conn.assigns[:digest])
+        else
+          conn
+        end
+
+      assign(conn, :valid_signature, HTTPSignatures.validate_conn(conn))
+    else
+      Logger.debug("No signature header!")
+      conn
     end
   end
 end

lib/pleroma/plugs/mapped_signature_to_identity_plug.ex

@@ -0,0 +1,70 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Plugs.MappedSignatureToIdentityPlug do
+  alias Pleroma.Signature
+  alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.Utils
+
+  import Plug.Conn
+  require Logger
+
+  def init(options), do: options
+
+  defp key_id_from_conn(conn) do
+    with %{"keyId" => key_id} <- HTTPSignatures.signature_for_conn(conn) do
+      Signature.key_id_to_actor_id(key_id)
+    else
+      _ ->
+        nil
+    end
+  end
+
+  defp user_from_key_id(conn) do
+    with key_actor_id when is_binary(key_actor_id) <- key_id_from_conn(conn),
+         {:ok, %User{} = user} <- User.get_or_fetch_by_ap_id(key_actor_id) do
+      user
+    else
+      _ ->
+        nil
+    end
+  end
+
+  def call(%{assigns: %{user: _}} = conn, _opts), do: conn
+
+  # if this has payload make sure it is signed by the same actor that made it
+  def call(%{assigns: %{valid_signature: true}, params: %{"actor" => actor}} = conn, _opts) do
+    with actor_id <- Utils.get_ap_id(actor),
+         {:user, %User{} = user} <- {:user, user_from_key_id(conn)},
+         {:user_match, true} <- {:user_match, user.ap_id == actor_id} do
+      assign(conn, :user, user)
+    else
+      {:user_match, false} ->
+        Logger.debug("Failed to map identity from signature (payload actor mismatch)")
+        Logger.debug("key_id=#{key_id_from_conn(conn)}, actor=#{actor}")
+        assign(conn, :valid_signature, false)
+
+      # remove me once testsuite uses mapped capabilities instead of what we do now
+      {:user, nil} ->
+        Logger.debug("Failed to map identity from signature (lookup failure)")
+        Logger.debug("key_id=#{key_id_from_conn(conn)}, actor=#{actor}")
+        conn
+    end
+  end
+
+  # no payload, probably a signed fetch
+  def call(%{assigns: %{valid_signature: true}} = conn, _opts) do
+    with %User{} = user <- user_from_key_id(conn) do
+      assign(conn, :user, user)
+    else
+      _ ->
+        Logger.debug("Failed to map identity from signature (no payload actor mismatch)")
+        Logger.debug("key_id=#{key_id_from_conn(conn)}")
+        assign(conn, :valid_signature, false)
+    end
+  end
+
+  # no signature at all
+  def call(conn, _opts), do: conn
+end

lib/pleroma/plugs/set_format_plug.ex

@@ -0,0 +1,24 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.SetFormatPlug do
+  import Plug.Conn, only: [assign: 3, fetch_query_params: 1]
+
+  def init(_), do: nil
+
+  def call(conn, _) do
+    case get_format(conn) do
+      nil -> conn
+      format -> assign(conn, :format, format)
+    end
+  end
+
+  defp get_format(conn) do
+    conn.private[:phoenix_format] ||
+      case fetch_query_params(conn) do
+        %{query_params: %{"_format" => format}} -> format
+        _ -> nil
+      end
+  end
+end

lib/pleroma/signature.ex

@@ -9,10 +9,19 @@ defmodule Pleroma.Signature do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
 
-  defp key_id_to_actor_id(key_id) do
-    URI.parse(key_id)
-    |> Map.put(:fragment, nil)
-    |> URI.to_string()
+  def key_id_to_actor_id(key_id) do
+    uri =
+      URI.parse(key_id)
+      |> Map.put(:fragment, nil)
+
+    uri =
+      if not is_nil(uri.path) and String.ends_with?(uri.path, "/publickey") do
+        Map.put(uri, :path, String.replace(uri.path, "/publickey", ""))
+      else
+        uri
+      end
+
+    URI.to_string(uri)
   end
 
   def fetch_public_key(conn) do

lib/pleroma/upload.ex

@@ -228,7 +228,14 @@ defmodule Pleroma.Upload do
           ""
         end
 
-    [base_url, "media", path]
+    prefix =
+      if is_nil(Pleroma.Config.get([__MODULE__, :base_url])) do
+        "media"
+      else
+        ""
+      end
+
+    [base_url, prefix, path]
     |> Path.join()
   end
 

lib/pleroma/user.ex

@@ -226,6 +226,7 @@ defmodule Pleroma.User do
     |> put_password_hash
   end
 
+  @spec reset_password(User.t(), map) :: {:ok, User.t()} | {:error, Ecto.Changeset.t()}
   def reset_password(%User{id: user_id} = user, data) do
     multi =
       Multi.new()
@@ -330,6 +331,7 @@ defmodule Pleroma.User do
 
   def needs_update?(_), do: true
 
+  @spec maybe_direct_follow(User.t(), User.t()) :: {:ok, User.t()} | {:error, String.t()}
   def maybe_direct_follow(%User{} = follower, %User{local: true, info: %{locked: true}}) do
     {:ok, follower}
   end
@@ -471,7 +473,7 @@ defmodule Pleroma.User do
   end
 
   def update_and_set_cache(changeset) do
-    with {:ok, user} <- Repo.update(changeset) do
+    with {:ok, user} <- Repo.update(changeset, stale_error_field: :id) do
       set_cache(user)
     else
       e -> e
@@ -586,12 +588,23 @@ defmodule Pleroma.User do
   @spec get_followers_query(User.t()) :: Ecto.Query.t()
   def get_followers_query(user), do: get_followers_query(user, nil)
 
+  @spec get_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
   def get_followers(user, page \\ nil) do
     q = get_followers_query(user, page)
 
     {:ok, Repo.all(q)}
   end
 
+  @spec get_external_followers(User.t(), pos_integer()) :: {:ok, list(User.t())}
+  def get_external_followers(user, page \\ nil) do
+    q =
+      user
+      |> get_followers_query(page)
+      |> User.Query.build(%{external: true})
+
+    {:ok, Repo.all(q)}
+  end
+
   def get_followers_ids(user, page \\ nil) do
     q = get_followers_query(user, page)
 
@@ -871,14 +884,26 @@ defmodule Pleroma.User do
   def muted_notifications?(user, %{ap_id: ap_id}),
     do: Enum.member?(user.info.muted_notifications, ap_id)
 
-  def blocks?(%User{info: info} = _user, %{ap_id: ap_id}) do
-    blocks = info.blocks
-    domain_blocks = info.domain_blocks
-    %{host: host} = URI.parse(ap_id)
-
-    Enum.member?(blocks, ap_id) || Enum.any?(domain_blocks, &(&1 == host))
+  def blocks?(%User{} = user, %User{} = target) do
+    blocks_ap_id?(user, target) || blocks_domain?(user, target)
   end
 
+  def blocks?(nil, _), do: false
+
+  def blocks_ap_id?(%User{} = user, %User{} = target) do
+    Enum.member?(user.info.blocks, target.ap_id)
+  end
+
+  def blocks_ap_id?(_, _), do: false
+
+  def blocks_domain?(%User{} = user, %User{} = target) do
+    domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks)
+    %{host: host} = URI.parse(target.ap_id)
+    Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, host)
+  end
+
+  def blocks_domain?(_, _), do: false
+
   def subscribed_to?(user, %{ap_id: ap_id}) do
     with %User{} = target <- get_cached_by_ap_id(ap_id) do
       Enum.member?(target.info.subscribers, user.ap_id)
@@ -1211,7 +1236,7 @@ defmodule Pleroma.User do
     data
     |> Map.put(:name, blank?(data[:name]) || data[:nickname])
     |> remote_user_creation()
-    |> Repo.insert(on_conflict: :replace_all, conflict_target: :nickname)
+    |> Repo.insert(on_conflict: :replace_all_except_primary_key, conflict_target: :nickname)
     |> set_cache()
   end
 

lib/pleroma/user_invite_token.ex

@@ -74,7 +74,7 @@ defmodule Pleroma.UserInviteToken do
 
   @spec find_by_token(token()) :: {:ok, UserInviteToken.t()} | nil
   def find_by_token(token) do
-    with invite <- Repo.get_by(UserInviteToken, token: token) do
+    with %UserInviteToken{} = invite <- Repo.get_by(UserInviteToken, token: token) do
       {:ok, invite}
     end
   end

lib/pleroma/web/activity_pub/activity_pub.ex

@@ -23,6 +23,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   import Pleroma.Web.ActivityPub.Visibility
 
   require Logger
+  require Pleroma.Constants
 
   # For Announce activities, we filter the recipients based on following status for any actors
   # that match actual users.  See issue #164 for more information about why this is necessary.
@@ -207,8 +208,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   def stream_out_participations(_, _), do: :noop
 
   def stream_out(activity) do
-    public = "https://www.w3.org/ns/activitystreams#Public"
-
     if activity.data["type"] in ["Create", "Announce", "Delete"] do
       object = Object.normalize(activity)
       # Do not stream out poll replies
@@ -216,7 +215,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
         Pleroma.Web.Streamer.stream("user", activity)
         Pleroma.Web.Streamer.stream("list", activity)
 
-        if Enum.member?(activity.data["to"], public) do
+        if get_visibility(activity) == "public" do
           Pleroma.Web.Streamer.stream("public", activity)
 
           if activity.local do
@@ -238,13 +237,8 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
             end
           end
         else
-          # TODO: Write test, replace with visibility test
-          if !Enum.member?(activity.data["cc"] || [], public) &&
-               !Enum.member?(
-                 activity.data["to"],
-                 User.get_cached_by_ap_id(activity.data["actor"]).follower_address
-               ),
-             do: Pleroma.Web.Streamer.stream("direct", activity)
+          if get_visibility(activity) == "direct",
+            do: Pleroma.Web.Streamer.stream("direct", activity)
         end
       end
     end
@@ -514,7 +508,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   end
 
   defp fetch_activities_for_context_query(context, opts) do
-    public = ["https://www.w3.org/ns/activitystreams#Public"]
+    public = [Pleroma.Constants.as_public()]
 
     recipients =
       if opts["user"], do: [opts["user"].ap_id | opts["user"].following] ++ public, else: public
@@ -555,7 +549,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
   end
 
   def fetch_public_activities(opts \\ %{}) do
-    q = fetch_activities_query(["https://www.w3.org/ns/activitystreams#Public"], opts)
+    q = fetch_activities_query([Pleroma.Constants.as_public()], opts)
 
     q
     |> restrict_unlisted()
@@ -631,17 +625,27 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
       |> Map.put("pinned_activity_ids", user.info.pinned_activities)
 
     recipients =
-      if reading_user do
-        ["https://www.w3.org/ns/activitystreams#Public"] ++
-          [reading_user.ap_id | reading_user.following]
-      else
-        ["https://www.w3.org/ns/activitystreams#Public"]
-      end
+      user_activities_recipients(%{
+        "godmode" => params["godmode"],
+        "reading_user" => reading_user
+      })
 
     fetch_activities(recipients, params)
     |> Enum.reverse()
   end
 
+  defp user_activities_recipients(%{"godmode" => true}) do
+    []
+  end
+
+  defp user_activities_recipients(%{"reading_user" => reading_user}) do
+    if reading_user do
+      [Pleroma.Constants.as_public()] ++ [reading_user.ap_id | reading_user.following]
+    else
+      [Pleroma.Constants.as_public()]
+    end
+  end
+
   defp restrict_since(query, %{"since_id" => ""}), do: query
 
   defp restrict_since(query, %{"since_id" => since_id}) do
@@ -823,7 +827,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
         fragment(
           "not (coalesce(?->'cc', '{}'::jsonb) \\?| ?)",
           activity.data,
-          ^["https://www.w3.org/ns/activitystreams#Public"]
+          ^[Pleroma.Constants.as_public()]
         )
     )
   end
@@ -960,7 +964,7 @@ defmodule Pleroma.Web.ActivityPub.ActivityPub do
       where:
         fragment("? && ?", activity.recipients, ^recipients) or
           (fragment("? && ?", activity.recipients, ^recipients_with_public) and
-             "https://www.w3.org/ns/activitystreams#Public" in activity.recipients)
+             ^Pleroma.Constants.as_public() in activity.recipients)
     )
   end
 

lib/pleroma/web/activity_pub/mrf.ex

@@ -25,4 +25,14 @@ defmodule Pleroma.Web.ActivityPub.MRF do
   defp get_policies(policy) when is_atom(policy), do: [policy]
   defp get_policies(policies) when is_list(policies), do: policies
   defp get_policies(_), do: []
+
+  @spec subdomains_regex([String.t()]) :: [Regex.t()]
+  def subdomains_regex(domains) when is_list(domains) do
+    for domain <- domains, do: ~r(^#{String.replace(domain, "*.", "(.*\\.)*")}$)
+  end
+
+  @spec subdomain_match?([Regex.t()], String.t()) :: boolean()
+  def subdomain_match?(domains, host) do
+    Enum.any?(domains, fn domain -> Regex.match?(domain, host) end)
+  end
 end

lib/pleroma/web/activity_pub/mrf/hellthread_policy.ex

@@ -4,6 +4,9 @@
 
 defmodule Pleroma.Web.ActivityPub.MRF.HellthreadPolicy do
   alias Pleroma.User
+
+  require Pleroma.Constants
+
   @moduledoc "Block messages with too much mentions (configurable)"
 
   @behaviour Pleroma.Web.ActivityPub.MRF
@@ -19,12 +22,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.HellthreadPolicy do
         when follower_collection? and recipients > threshold ->
           message
           |> Map.put("to", [follower_collection])
-          |> Map.put("cc", ["https://www.w3.org/ns/activitystreams#Public"])
+          |> Map.put("cc", [Pleroma.Constants.as_public()])
 
         {:public, recipients} when recipients > threshold ->
           message
           |> Map.put("to", [])
-          |> Map.put("cc", ["https://www.w3.org/ns/activitystreams#Public"])
+          |> Map.put("cc", [Pleroma.Constants.as_public()])
 
         _ ->
           message
@@ -51,10 +54,10 @@ defmodule Pleroma.Web.ActivityPub.MRF.HellthreadPolicy do
     recipients = (message["to"] || []) ++ (message["cc"] || [])
     follower_collection = User.get_cached_by_ap_id(message["actor"]).follower_address
 
-    if Enum.member?(recipients, "https://www.w3.org/ns/activitystreams#Public") do
+    if Enum.member?(recipients, Pleroma.Constants.as_public()) do
       recipients =
         recipients
-        |> List.delete("https://www.w3.org/ns/activitystreams#Public")
+        |> List.delete(Pleroma.Constants.as_public())
         |> List.delete(follower_collection)
 
       {:public, length(recipients)}

lib/pleroma/web/activity_pub/mrf/keyword_policy.ex

@@ -3,6 +3,8 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 defmodule Pleroma.Web.ActivityPub.MRF.KeywordPolicy do
+  require Pleroma.Constants
+
   @moduledoc "Reject or Word-Replace messages with a keyword or regex"
 
   @behaviour Pleroma.Web.ActivityPub.MRF
@@ -31,12 +33,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.KeywordPolicy do
   defp check_ftl_removal(
          %{"to" => to, "object" => %{"content" => content, "summary" => summary}} = message
        ) do
-    if "https://www.w3.org/ns/activitystreams#Public" in to and
+    if Pleroma.Constants.as_public() in to and
          Enum.any?(Pleroma.Config.get([:mrf_keyword, :federated_timeline_removal]), fn pattern ->
            string_matches?(content, pattern) or string_matches?(summary, pattern)
          end) do
-      to = List.delete(to, "https://www.w3.org/ns/activitystreams#Public")
-      cc = ["https://www.w3.org/ns/activitystreams#Public" | message["cc"] || []]
+      to = List.delete(to, Pleroma.Constants.as_public())
+      cc = [Pleroma.Constants.as_public() | message["cc"] || []]
 
       message =
         message

lib/pleroma/web/activity_pub/mrf/reject_non_public.ex

@@ -10,7 +10,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.RejectNonPublic do
 
   @behaviour Pleroma.Web.ActivityPub.MRF
 
-  @public "https://www.w3.org/ns/activitystreams#Public"
+  require Pleroma.Constants
 
   @impl true
   def filter(%{"type" => "Create"} = object) do
@@ -19,8 +19,8 @@ defmodule Pleroma.Web.ActivityPub.MRF.RejectNonPublic do
     # Determine visibility
     visibility =
       cond do
-        @public in object["to"] -> "public"
-        @public in object["cc"] -> "unlisted"
+        Pleroma.Constants.as_public() in object["to"] -> "public"
+        Pleroma.Constants.as_public() in object["cc"] -> "unlisted"
         user.follower_address in object["to"] -> "followers"
         true -> "direct"
       end

lib/pleroma/web/activity_pub/mrf/simple_policy.ex

@@ -4,22 +4,31 @@
 
 defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   alias Pleroma.User
+  alias Pleroma.Web.ActivityPub.MRF
   @moduledoc "Filter activities depending on their origin instance"
-  @behaviour Pleroma.Web.ActivityPub.MRF
+  @behaviour MRF
+
+  require Pleroma.Constants
 
   defp check_accept(%{host: actor_host} = _actor_info, object) do
-    accepts = Pleroma.Config.get([:mrf_simple, :accept])
+    accepts =
+      Pleroma.Config.get([:mrf_simple, :accept])
+      |> MRF.subdomains_regex()
 
     cond do
       accepts == [] -> {:ok, object}
       actor_host == Pleroma.Config.get([Pleroma.Web.Endpoint, :url, :host]) -> {:ok, object}
-      Enum.member?(accepts, actor_host) -> {:ok, object}
+      MRF.subdomain_match?(accepts, actor_host) -> {:ok, object}
       true -> {:reject, nil}
     end
   end
 
   defp check_reject(%{host: actor_host} = _actor_info, object) do
-    if Enum.member?(Pleroma.Config.get([:mrf_simple, :reject]), actor_host) do
+    rejects =
+      Pleroma.Config.get([:mrf_simple, :reject])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(rejects, actor_host) do
       {:reject, nil}
     else
       {:ok, object}
@@ -31,8 +40,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
          %{"type" => "Create", "object" => %{"attachment" => child_attachment}} = object
        )
        when length(child_attachment) > 0 do
+    media_removal =
+      Pleroma.Config.get([:mrf_simple, :media_removal])
+      |> MRF.subdomains_regex()
+
     object =
-      if Enum.member?(Pleroma.Config.get([:mrf_simple, :media_removal]), actor_host) do
+      if MRF.subdomain_match?(media_removal, actor_host) do
         child_object = Map.delete(object["object"], "attachment")
         Map.put(object, "object", child_object)
       else
@@ -51,8 +64,12 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
            "object" => child_object
          } = object
        ) do
+    media_nsfw =
+      Pleroma.Config.get([:mrf_simple, :media_nsfw])
+      |> MRF.subdomains_regex()
+
     object =
-      if Enum.member?(Pleroma.Config.get([:mrf_simple, :media_nsfw]), actor_host) do
+      if MRF.subdomain_match?(media_nsfw, actor_host) do
         tags = (child_object["tag"] || []) ++ ["nsfw"]
         child_object = Map.put(child_object, "tag", tags)
         child_object = Map.put(child_object, "sensitive", true)
@@ -67,21 +84,17 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   defp check_media_nsfw(_actor_info, object), do: {:ok, object}
 
   defp check_ftl_removal(%{host: actor_host} = _actor_info, object) do
-    object =
-      with true <-
-             Enum.member?(
-               Pleroma.Config.get([:mrf_simple, :federated_timeline_removal]),
-               actor_host
-             ),
-           user <- User.get_cached_by_ap_id(object["actor"]),
-           true <- "https://www.w3.org/ns/activitystreams#Public" in object["to"] do
-        to =
-          List.delete(object["to"], "https://www.w3.org/ns/activitystreams#Public") ++
-            [user.follower_address]
+    timeline_removal =
+      Pleroma.Config.get([:mrf_simple, :federated_timeline_removal])
+      |> MRF.subdomains_regex()
 
-        cc =
-          List.delete(object["cc"], user.follower_address) ++
-            ["https://www.w3.org/ns/activitystreams#Public"]
+    object =
+      with true <- MRF.subdomain_match?(timeline_removal, actor_host),
+           user <- User.get_cached_by_ap_id(object["actor"]),
+           true <- Pleroma.Constants.as_public() in object["to"] do
+        to = List.delete(object["to"], Pleroma.Constants.as_public()) ++ [user.follower_address]
+
+        cc = List.delete(object["cc"], user.follower_address) ++ [Pleroma.Constants.as_public()]
 
         object
         |> Map.put("to", to)
@@ -94,7 +107,11 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   end
 
   defp check_report_removal(%{host: actor_host} = _actor_info, %{"type" => "Flag"} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :report_removal]) do
+    report_removal =
+      Pleroma.Config.get([:mrf_simple, :report_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(report_removal, actor_host) do
       {:reject, nil}
     else
       {:ok, object}
@@ -104,7 +121,11 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   defp check_report_removal(_actor_info, object), do: {:ok, object}
 
   defp check_avatar_removal(%{host: actor_host} = _actor_info, %{"icon" => _icon} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :avatar_removal]) do
+    avatar_removal =
+      Pleroma.Config.get([:mrf_simple, :avatar_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(avatar_removal, actor_host) do
       {:ok, Map.delete(object, "icon")}
     else
       {:ok, object}
@@ -114,7 +135,11 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicy do
   defp check_avatar_removal(_actor_info, object), do: {:ok, object}
 
   defp check_banner_removal(%{host: actor_host} = _actor_info, %{"image" => _image} = object) do
-    if actor_host in Pleroma.Config.get([:mrf_simple, :banner_removal]) do
+    banner_removal =
+      Pleroma.Config.get([:mrf_simple, :banner_removal])
+      |> MRF.subdomains_regex()
+
+    if MRF.subdomain_match?(banner_removal, actor_host) do
       {:ok, Map.delete(object, "image")}
     else
       {:ok, object}

lib/pleroma/web/activity_pub/mrf/tag_policy.ex

@@ -19,7 +19,7 @@ defmodule Pleroma.Web.ActivityPub.MRF.TagPolicy do
      - `mrf_tag:disable-any-subscription`: Reject any follow requests
   """
 
-  @public "https://www.w3.org/ns/activitystreams#Public"
+  require Pleroma.Constants
 
   defp get_tags(%User{tags: tags}) when is_list(tags), do: tags
   defp get_tags(_), do: []
@@ -70,9 +70,9 @@ defmodule Pleroma.Web.ActivityPub.MRF.TagPolicy do
        ) do
     user = User.get_cached_by_ap_id(actor)
 
-    if Enum.member?(to, @public) do
-      to = List.delete(to, @public) ++ [user.follower_address]
-      cc = List.delete(cc, user.follower_address) ++ [@public]
+    if Enum.member?(to, Pleroma.Constants.as_public()) do
+      to = List.delete(to, Pleroma.Constants.as_public()) ++ [user.follower_address]
+      cc = List.delete(cc, user.follower_address) ++ [Pleroma.Constants.as_public()]
 
       object =
         object
@@ -103,9 +103,10 @@ defmodule Pleroma.Web.ActivityPub.MRF.TagPolicy do
        ) do
     user = User.get_cached_by_ap_id(actor)
 
-    if Enum.member?(to, @public) or Enum.member?(cc, @public) do
-      to = List.delete(to, @public) ++ [user.follower_address]
-      cc = List.delete(cc, @public)
+    if Enum.member?(to, Pleroma.Constants.as_public()) or
+         Enum.member?(cc, Pleroma.Constants.as_public()) do
+      to = List.delete(to, Pleroma.Constants.as_public()) ++ [user.follower_address]
+      cc = List.delete(cc, Pleroma.Constants.as_public())
 
       object =
         object

lib/pleroma/web/activity_pub/publisher.ex

@@ -11,6 +11,8 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
   alias Pleroma.Web.ActivityPub.Relay
   alias Pleroma.Web.ActivityPub.Transmogrifier
 
+  require Pleroma.Constants
+
   import Pleroma.Web.ActivityPub.Visibility
 
   @behaviour Pleroma.Web.Federator.Publisher
@@ -87,18 +89,23 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
     if public do
       true
     else
-      inbox_info = URI.parse(inbox)
-      !Enum.member?(Config.get([:instance, :quarantined_instances], []), inbox_info.host)
+      %{host: host} = URI.parse(inbox)
+
+      quarantined_instances =
+        Config.get([:instance, :quarantined_instances], [])
+        |> Pleroma.Web.ActivityPub.MRF.subdomains_regex()
+
+      !Pleroma.Web.ActivityPub.MRF.subdomain_match?(quarantined_instances, host)
     end
   end
 
+  @spec recipients(User.t(), Activity.t()) :: list(User.t()) | []
   defp recipients(actor, activity) do
-    followers =
+    {:ok, followers} =
       if actor.follower_address in activity.recipients do
-        {:ok, followers} = User.get_followers(actor)
-        Enum.filter(followers, &(!&1.local))
+        User.get_external_followers(actor)
       else
-        []
+        {:ok, []}
       end
 
     Pleroma.Web.Salmon.remote_users(actor, activity) ++ followers
@@ -112,6 +119,43 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
     |> Enum.map(& &1.ap_id)
   end
 
+  defp maybe_use_sharedinbox(%User{info: %{source_data: data}}),
+    do: (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+
+  @doc """
+  Determine a user inbox to use based on heuristics.  These heuristics
+  are based on an approximation of the ``sharedInbox`` rules in the
+  [ActivityPub specification][ap-sharedinbox].
+
+  Please do not edit this function (or its children) without reading
+  the spec, as editing the code is likely to introduce some breakage
+  without some familiarity.
+
+     [ap-sharedinbox]: https://www.w3.org/TR/activitypub/#shared-inbox-delivery
+  """
+  def determine_inbox(
+        %Activity{data: activity_data},
+        %User{info: %{source_data: data}} = user
+      ) do
+    to = activity_data["to"] || []
+    cc = activity_data["cc"] || []
+    type = activity_data["type"]
+
+    cond do
+      type == "Delete" ->
+        maybe_use_sharedinbox(user)
+
+      Pleroma.Constants.as_public() in to || Pleroma.Constants.as_public() in cc ->
+        maybe_use_sharedinbox(user)
+
+      length(to) + length(cc) > 1 ->
+        maybe_use_sharedinbox(user)
+
+      true ->
+        data["inbox"]
+    end
+  end
+
   @doc """
   Publishes an activity with BCC to all relevant peers.
   """
@@ -166,8 +210,8 @@ defmodule Pleroma.Web.ActivityPub.Publisher do
 
     recipients(actor, activity)
     |> Enum.filter(fn user -> User.ap_enabled?(user) end)
-    |> Enum.map(fn %{info: %{source_data: data}} ->
-      (is_map(data["endpoints"]) && Map.get(data["endpoints"], "sharedInbox")) || data["inbox"]
+    |> Enum.map(fn %User{} = user ->
+      determine_inbox(activity, user)
     end)
     |> Enum.uniq()
     |> Enum.filter(fn inbox -> should_federate?(inbox, public) end)

lib/pleroma/web/activity_pub/transmogrifier.ex

@@ -19,6 +19,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
   import Ecto.Query
 
   require Logger
+  require Pleroma.Constants
 
   @doc """
   Modifies an incoming AP object (mastodon format) to our internal format.
@@ -102,8 +103,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
 
     follower_collection = User.get_cached_by_ap_id(Containment.get_actor(object)).follower_address
 
-    explicit_mentions =
-      explicit_mentions ++ ["https://www.w3.org/ns/activitystreams#Public", follower_collection]
+    explicit_mentions = explicit_mentions ++ [Pleroma.Constants.as_public(), follower_collection]
 
     fix_explicit_addressing(object, explicit_mentions, follower_collection)
   end
@@ -115,11 +115,11 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
 
     if followers_collection not in recipients do
       cond do
-        "https://www.w3.org/ns/activitystreams#Public" in cc ->
+        Pleroma.Constants.as_public() in cc ->
           to = to ++ [followers_collection]
           Map.put(object, "to", to)
 
-        "https://www.w3.org/ns/activitystreams#Public" in to ->
+        Pleroma.Constants.as_public() in to ->
           cc = cc ++ [followers_collection]
           Map.put(object, "cc", cc)
 
@@ -480,8 +480,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
          {:ok, %User{} = follower} <- User.get_or_fetch_by_ap_id(follower),
          {:ok, activity} <- ActivityPub.follow(follower, followed, id, false) do
       with deny_follow_blocked <- Pleroma.Config.get([:user, :deny_follow_blocked]),
-           {_, false} <-
-             {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked},
+           {_, false} <- {:user_blocked, User.blocks?(followed, follower) && deny_follow_blocked},
            {_, false} <- {:user_locked, User.locked?(followed)},
            {_, {:ok, follower}} <- {:follow, User.follow(follower, followed)},
            {_, {:ok, _}} <-
@@ -656,20 +655,7 @@ defmodule Pleroma.Web.ActivityPub.Transmogrifier do
       nil ->
         case User.get_cached_by_ap_id(object_id) do
           %User{ap_id: ^actor} = user ->
-            {:ok, followers} = User.get_followers(user)
-
-            Enum.each(followers, fn follower ->
-              User.unfollow(follower, user)
-            end)
-
-            {:ok, friends} = User.get_friends(user)
-
-            Enum.each(friends, fn followed ->
-              User.unfollow(user, followed)
-            end)
-
-            User.invalidate_cache(user)
-            Repo.delete(user)
+            User.delete(user)
 
           nil ->
             :error

lib/pleroma/web/activity_pub/utils.ex

@@ -18,6 +18,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
   import Ecto.Query
 
   require Logger
+  require Pleroma.Constants
 
   @supported_object_types ["Article", "Note", "Video", "Page", "Question", "Answer"]
   @supported_report_states ~w(open closed resolved)
@@ -418,7 +419,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
       "type" => "Follow",
       "actor" => follower_id,
       "to" => [followed_id],
-      "cc" => ["https://www.w3.org/ns/activitystreams#Public"],
+      "cc" => [Pleroma.Constants.as_public()],
       "object" => followed_id,
       "state" => "pending"
     }
@@ -510,7 +511,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
       "actor" => ap_id,
       "object" => id,
       "to" => [user.follower_address, object.data["actor"]],
-      "cc" => ["https://www.w3.org/ns/activitystreams#Public"],
+      "cc" => [Pleroma.Constants.as_public()],
       "context" => object.data["context"]
     }
 
@@ -530,7 +531,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
       "actor" => ap_id,
       "object" => activity.data,
       "to" => [user.follower_address, activity.data["actor"]],
-      "cc" => ["https://www.w3.org/ns/activitystreams#Public"],
+      "cc" => [Pleroma.Constants.as_public()],
       "context" => context
     }
 
@@ -547,7 +548,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
       "actor" => ap_id,
       "object" => activity.data,
       "to" => [user.follower_address, activity.data["actor"]],
-      "cc" => ["https://www.w3.org/ns/activitystreams#Public"],
+      "cc" => [Pleroma.Constants.as_public()],
       "context" => context
     }
 
@@ -556,7 +557,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
 
   def add_announce_to_object(
         %Activity{
-          data: %{"actor" => actor, "cc" => ["https://www.w3.org/ns/activitystreams#Public"]}
+          data: %{"actor" => actor, "cc" => [Pleroma.Constants.as_public()]}
         },
         object
       ) do
@@ -765,7 +766,7 @@ defmodule Pleroma.Web.ActivityPub.Utils do
        ) do
     cc = Map.get(data, "cc", [])
     follower_address = User.get_cached_by_ap_id(data["actor"]).follower_address
-    public = "https://www.w3.org/ns/activitystreams#Public"
+    public = Pleroma.Constants.as_public()
 
     case visibility do
       "public" ->

lib/pleroma/web/activity_pub/visibility.ex

@@ -8,14 +8,14 @@ defmodule Pleroma.Web.ActivityPub.Visibility do
   alias Pleroma.Repo
   alias Pleroma.User
 
+  require Pleroma.Constants
+
+  @spec is_public?(Object.t() | Activity.t() | map()) :: boolean()
   def is_public?(%Object{data: %{"type" => "Tombstone"}}), do: false
   def is_public?(%Object{data: data}), do: is_public?(data)
   def is_public?(%Activity{data: data}), do: is_public?(data)
   def is_public?(%{"directMessage" => true}), do: false
-
-  def is_public?(data) do
-    "https://www.w3.org/ns/activitystreams#Public" in (data["to"] ++ (data["cc"] || []))
-  end
+  def is_public?(data), do: Pleroma.Constants.as_public() in (data["to"] ++ (data["cc"] || []))
 
   def is_private?(activity) do
     with false <- is_public?(activity),
@@ -69,15 +69,14 @@ defmodule Pleroma.Web.ActivityPub.Visibility do
   end
 
   def get_visibility(object) do
-    public = "https://www.w3.org/ns/activitystreams#Public"
     to = object.data["to"] || []
     cc = object.data["cc"] || []
 
     cond do
-      public in to ->
+      Pleroma.Constants.as_public() in to ->
         "public"
 
-      public in cc ->
+      Pleroma.Constants.as_public() in cc ->
         "unlisted"
 
       # this should use the sql for the object's activity

lib/pleroma/web/admin_api/admin_api_controller.ex

@@ -82,6 +82,25 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
     end
   end
 
+  def list_user_statuses(conn, %{"nickname" => nickname} = params) do
+    godmode = params["godmode"] == "true" || params["godmode"] == true
+
+    with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
+      {_, page_size} = page_params(params)
+
+      activities =
+        ActivityPub.fetch_user_activities(user, nil, %{
+          "limit" => page_size,
+          "godmode" => godmode
+        })
+
+      conn
+      |> json(StatusView.render("index.json", %{activities: activities, as: :activity}))
+    else
+      _ -> {:error, :not_found}
+    end
+  end
+
   def user_toggle_activation(conn, %{"nickname" => nickname}) do
     user = User.get_cached_by_nickname(nickname)
 
@@ -272,11 +291,13 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
 
   @doc "Revokes invite by token"
   def revoke_invite(conn, %{"token" => token}) do
-    invite = UserInviteToken.find_by_token!(token)
-    {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true})
-
-    conn
-    |> json(AccountView.render("invite.json", %{invite: updated_invite}))
+    with {:ok, invite} <- UserInviteToken.find_by_token(token),
+         {:ok, updated_invite} = UserInviteToken.update_invite(invite, %{used: true}) do
+      conn
+      |> json(AccountView.render("invite.json", %{invite: updated_invite}))
+    else
+      nil -> {:error, :not_found}
+    end
   end
 
   @doc "Get a password reset token (base64 string) for given nickname"
@@ -358,6 +379,16 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do
     end
   end
 
+  def migrate_to_db(conn, _params) do
+    Mix.Tasks.Pleroma.Config.run(["migrate_to_db"])
+    json(conn, %{})
+  end
+
+  def migrate_from_db(conn, _params) do
+    Mix.Tasks.Pleroma.Config.run(["migrate_from_db", Pleroma.Config.get(:env), "true"])
+    json(conn, %{})
+  end
+
   def config_show(conn, _params) do
     configs = Pleroma.Repo.all(Config)
 

lib/pleroma/web/admin_api/config.ex

@@ -84,6 +84,7 @@ defmodule Pleroma.Web.AdminAPI.Config do
   end
 
   defp do_convert({:dispatch, [entity]}), do: %{"tuple" => [":dispatch", [inspect(entity)]]}
+  defp do_convert({:partial_chain, entity}), do: %{"tuple" => [":partial_chain", inspect(entity)]}
 
   defp do_convert(entity) when is_tuple(entity),
     do: %{"tuple" => do_convert(Tuple.to_list(entity))}
@@ -113,11 +114,15 @@ defmodule Pleroma.Web.AdminAPI.Config do
   defp do_transform(%Regex{} = entity) when is_map(entity), do: entity
 
   defp do_transform(%{"tuple" => [":dispatch", [entity]]}) do
-    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
-    {dispatch_settings, []} = Code.eval_string(cleaned_string, [], requires: [], macros: [])
+    {dispatch_settings, []} = do_eval(entity)
     {:dispatch, [dispatch_settings]}
   end
 
+  defp do_transform(%{"tuple" => [":partial_chain", entity]}) do
+    {partial_chain, []} = do_eval(entity)
+    {:partial_chain, partial_chain}
+  end
+
   defp do_transform(%{"tuple" => entity}) do
     Enum.reduce(entity, {}, fn val, acc -> Tuple.append(acc, do_transform(val)) end)
   end
@@ -149,4 +154,9 @@ defmodule Pleroma.Web.AdminAPI.Config do
       do: String.to_existing_atom("Elixir." <> value),
       else: value
   end
+
+  defp do_eval(entity) do
+    cleaned_string = String.replace(entity, ~r/[^\w|^{:,[|^,|^[|^\]^}|^\/|^\.|^"]^\s/, "")
+    Code.eval_string(cleaned_string, [], requires: [], macros: [])
+  end
 end

lib/pleroma/web/auth/authenticator.ex

@@ -21,8 +21,7 @@ defmodule Pleroma.Web.Auth.Authenticator do
   def create_from_registration(plug, registration),
     do: implementation().create_from_registration(plug, registration)
 
-  @callback get_registration(Plug.Conn.t()) ::
-              {:ok, Registration.t()} | {:error, any()}
+  @callback get_registration(Plug.Conn.t()) :: {:ok, Registration.t()} | {:error, any()}
   def get_registration(plug), do: implementation().get_registration(plug)
 
   @callback handle_error(Plug.Conn.t(), any()) :: any()

lib/pleroma/web/common_api/common_api.ex

@@ -300,8 +300,7 @@ defmodule Pleroma.Web.CommonAPI do
            }
          } = activity <- get_by_id_or_ap_id(id_or_ap_id),
          true <- Visibility.is_public?(activity),
-         %{valid?: true} = info_changeset <-
-           User.Info.add_pinnned_activity(user.info, activity),
+         %{valid?: true} = info_changeset <- User.Info.add_pinnned_activity(user.info, activity),
          changeset <-
            Ecto.Changeset.change(user) |> Ecto.Changeset.put_embed(:info, info_changeset),
          {:ok, _user} <- User.update_and_set_cache(changeset) do

lib/pleroma/web/common_api/utils.ex

@@ -19,11 +19,17 @@ defmodule Pleroma.Web.CommonAPI.Utils do
   alias Pleroma.Web.MediaProxy
 
   require Logger
+  require Pleroma.Constants
 
   # This is a hack for twidere.
   def get_by_id_or_ap_id(id) do
     activity =
-      Activity.get_by_id_with_object(id) || Activity.get_create_by_object_ap_id_with_object(id)
+      with true <- Pleroma.FlakeId.is_flake_id?(id),
+           %Activity{} = activity <- Activity.get_by_id_with_object(id) do
+        activity
+      else
+        _ -> Activity.get_create_by_object_ap_id_with_object(id)
+      end
 
     activity &&
       if activity.data["type"] == "Create" do
@@ -66,7 +72,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
   @spec get_to_and_cc(User.t(), list(String.t()), Activity.t() | nil, String.t()) ::
           {list(String.t()), list(String.t())}
   def get_to_and_cc(user, mentioned_users, inReplyTo, "public") do
-    to = ["https://www.w3.org/ns/activitystreams#Public" | mentioned_users]
+    to = [Pleroma.Constants.as_public() | mentioned_users]
     cc = [user.follower_address]
 
     if inReplyTo do
@@ -78,7 +84,7 @@ defmodule Pleroma.Web.CommonAPI.Utils do
 
   def get_to_and_cc(user, mentioned_users, inReplyTo, "unlisted") do
     to = [user.follower_address | mentioned_users]
-    cc = ["https://www.w3.org/ns/activitystreams#Public"]
+    cc = [Pleroma.Constants.as_public()]
 
     if inReplyTo do
       {Enum.uniq([inReplyTo.data["actor"] | to]), cc}
@@ -439,6 +445,13 @@ defmodule Pleroma.Web.CommonAPI.Utils do
 
   def maybe_notify_mentioned_recipients(recipients, _), do: recipients
 
+  # Do not notify subscribers if author is making a reply
+  def maybe_notify_subscribers(recipients, %Activity{
+        object: %Object{data: %{"inReplyTo" => _ap_id}}
+      }) do
+    recipients
+  end
+
   def maybe_notify_subscribers(
         recipients,
         %Activity{data: %{"actor" => actor, "type" => type}} = activity

lib/pleroma/web/fallback_redirect_controller.ex

@@ -0,0 +1,77 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Fallback.RedirectController do
+  use Pleroma.Web, :controller
+  require Logger
+  alias Pleroma.User
+  alias Pleroma.Web.Metadata
+
+  def api_not_implemented(conn, _params) do
+    conn
+    |> put_status(404)
+    |> json(%{error: "Not implemented"})
+  end
+
+  def redirector(conn, _params, code \\ 200)
+
+  # redirect to admin section
+  # /pleroma/admin -> /pleroma/admin/
+  #
+  def redirector(conn, %{"path" => ["pleroma", "admin"]} = _, _code) do
+    redirect(conn, to: "/pleroma/admin/")
+  end
+
+  def redirector(conn, _params, code) do
+    conn
+    |> put_resp_content_type("text/html")
+    |> send_file(code, index_file_path())
+  end
+
+  def redirector_with_meta(conn, %{"maybe_nickname_or_id" => maybe_nickname_or_id} = params) do
+    with %User{} = user <- User.get_cached_by_nickname_or_id(maybe_nickname_or_id) do
+      redirector_with_meta(conn, %{user: user})
+    else
+      nil ->
+        redirector(conn, params)
+    end
+  end
+
+  def redirector_with_meta(conn, params) do
+    {:ok, index_content} = File.read(index_file_path())
+
+    tags =
+      try do
+        Metadata.build_tags(params)
+      rescue
+        e ->
+          Logger.error(
+            "Metadata rendering for #{conn.request_path} failed.\n" <>
+              Exception.format(:error, e, __STACKTRACE__)
+          )
+
+          ""
+      end
+
+    response = String.replace(index_content, "<!--server-generated-meta-->", tags)
+
+    conn
+    |> put_resp_content_type("text/html")
+    |> send_resp(200, response)
+  end
+
+  def index_file_path do
+    Pleroma.Plugs.InstanceStatic.file_path("index.html")
+  end
+
+  def registration_page(conn, params) do
+    redirector(conn, params)
+  end
+
+  def empty(conn, _params) do
+    conn
+    |> put_status(204)
+    |> text("")
+  end
+end

lib/pleroma/web/mastodon_api/mastodon_api_controller.ex

@@ -4,6 +4,9 @@
 
 defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
   use Pleroma.Web, :controller
+
+  import Pleroma.Web.ControllerHelper, only: [json_response: 3]
+
   alias Ecto.Changeset
   alias Pleroma.Activity
   alias Pleroma.Bookmark
@@ -46,6 +49,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
   import Ecto.Query
 
   require Logger
+  require Pleroma.Constants
 
   @rate_limited_relations_actions ~w(follow unfollow)a
 
@@ -74,6 +78,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
   plug(RateLimiter, :app_account_creation when action == :account_register)
   plug(RateLimiter, :search when action in [:search, :search2, :account_search])
   plug(RateLimiter, :password_reset when action == :password_reset)
+  plug(RateLimiter, :account_confirmation_resend when action == :account_confirmation_resend)
 
   @local_mastodon_name "Mastodon-Local"
 
@@ -440,7 +445,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
   end
 
   def user_statuses(%{assigns: %{user: reading_user}} = conn, params) do
-    with %User{} = user <- User.get_cached_by_id(params["id"]) do
+    with %User{} = user <- User.get_cached_by_nickname_or_id(params["id"]) do
       params =
         params
         |> Map.put("tag", params["tagged"])
@@ -883,7 +888,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
     with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
          %Object{data: %{"likes" => likes}} <- Object.normalize(object) do
       q = from(u in User, where: u.ap_id in ^likes)
-      users = Repo.all(q)
+
+      users =
+        Repo.all(q)
+        |> Enum.filter(&(not User.blocks?(user, &1)))
 
       conn
       |> put_view(AccountView)
@@ -897,7 +905,10 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
     with %Activity{data: %{"object" => object}} <- Activity.get_by_id(id),
          %Object{data: %{"announcements" => announces}} <- Object.normalize(object) do
       q = from(u in User, where: u.ap_id in ^announces)
-      users = Repo.all(q)
+
+      users =
+        Repo.all(q)
+        |> Enum.filter(&(not User.blocks?(user, &1)))
 
       conn
       |> put_view(AccountView)
@@ -1214,10 +1225,9 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
 
       recipients =
         if for_user do
-          ["https://www.w3.org/ns/activitystreams#Public"] ++
-            [for_user.ap_id | for_user.following]
+          [Pleroma.Constants.as_public()] ++ [for_user.ap_id | for_user.following]
         else
-          ["https://www.w3.org/ns/activitystreams#Public"]
+          [Pleroma.Constants.as_public()]
         end
 
       activities =
@@ -1833,6 +1843,16 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIController do
     end
   end
 
+  def account_confirmation_resend(conn, params) do
+    nickname_or_email = params["email"] || params["nickname"]
+
+    with %User{} = user <- User.get_by_nickname_or_email(nickname_or_email),
+         {:ok, _} <- User.try_send_confirmation_email(user) do
+      conn
+      |> json_response(:no_content, "")
+    end
+  end
+
   def try_render(conn, target, params)
       when is_binary(target) do
     case render(conn, target, params) do

lib/pleroma/web/mastodon_api/views/account_view.ex

@@ -50,13 +50,13 @@ defmodule Pleroma.Web.MastodonAPI.AccountView do
       id: to_string(target.id),
       following: User.following?(user, target),
       followed_by: User.following?(target, user),
-      blocking: User.blocks?(user, target),
-      blocked_by: User.blocks?(target, user),
+      blocking: User.blocks_ap_id?(user, target),
+      blocked_by: User.blocks_ap_id?(target, user),
       muting: User.mutes?(user, target),
       muting_notifications: User.muted_notifications?(user, target),
       subscribing: User.subscribed_to?(user, target),
       requested: requested,
-      domain_blocking: false,
+      domain_blocking: User.blocks_domain?(user, target),
       showing_reblogs: User.showing_reblogs?(user, target),
       endorsed: false
     }

lib/pleroma/web/mastodon_api/views/status_view.ex

@@ -222,7 +222,7 @@ defmodule Pleroma.Web.MastodonAPI.StatusView do
       if user.local do
         Pleroma.Web.Router.Helpers.o_status_url(Pleroma.Web.Endpoint, :notice, activity)
       else
-        object.data["external_url"] || object.data["id"]
+        object.data["url"] || object.data["external_url"] || object.data["id"]
       end
 
     %{

lib/pleroma/web/nodeinfo/nodeinfo_controller.ex

@@ -165,6 +165,7 @@ defmodule Pleroma.Web.Nodeinfo.NodeinfoController do
         },
         accountActivationRequired: Config.get([:instance, :account_activation_required], false),
         invitesEnabled: Config.get([:instance, :invites_enabled], false),
+        mailerEnabled: Config.get([Pleroma.Emails.Mailer, :enabled], false),
         features: features,
         restrictedNicknames: Config.get([Pleroma.User, :restricted_nicknames]),
         skipThreadContainment: Config.get([:instance, :skip_thread_containment], false)

lib/pleroma/web/oauth/oauth_controller.ex

@@ -365,8 +365,7 @@ defmodule Pleroma.Web.OAuth.OAuthController do
   def register(%Plug.Conn{} = conn, %{"authorization" => _, "op" => "connect"} = params) do
     with registration_id when not is_nil(registration_id) <- get_session_registration_id(conn),
          %Registration{} = registration <- Repo.get(Registration, registration_id),
-         {_, {:ok, auth}} <-
-           {:create_authorization, do_create_authorization(conn, params)},
+         {_, {:ok, auth}} <- {:create_authorization, do_create_authorization(conn, params)},
          %User{} = user <- Repo.preload(auth, :user).user,
          {:ok, _updated_registration} <- Registration.bind_to_user(registration, user) do
       conn

lib/pleroma/web/oauth/token.ex

@@ -44,8 +44,7 @@ defmodule Pleroma.Web.OAuth.Token do
     |> Repo.find_resource()
   end
 
-  @spec exchange_token(App.t(), Authorization.t()) ::
-          {:ok, Token.t()} | {:error, Changeset.t()}
+  @spec exchange_token(App.t(), Authorization.t()) :: {:ok, Token.t()} | {:error, Changeset.t()}
   def exchange_token(app, auth) do
     with {:ok, auth} <- Authorization.use_token(auth),
          true <- auth.app_id == app.id do

lib/pleroma/web/ostatus/activity_representer.ex

@@ -9,6 +9,7 @@ defmodule Pleroma.Web.OStatus.ActivityRepresenter do
   alias Pleroma.Web.OStatus.UserRepresenter
 
   require Logger
+  require Pleroma.Constants
 
   defp get_href(id) do
     with %Object{data: %{"external_url" => external_url}} <- Object.get_cached_by_ap_id(id) do
@@ -34,7 +35,7 @@ defmodule Pleroma.Web.OStatus.ActivityRepresenter do
     Enum.map(to, fn id ->
       cond do
         # Special handling for the AP/Ostatus public collections
-        "https://www.w3.org/ns/activitystreams#Public" == id ->
+        Pleroma.Constants.as_public() == id ->
           {:link,
            [
              rel: "mentioned",

lib/pleroma/web/ostatus/handlers/follow_handler.ex

@@ -9,14 +9,18 @@ defmodule Pleroma.Web.OStatus.FollowHandler do
   alias Pleroma.Web.XML
 
   def handle(entry, doc) do
-    with {:ok, actor} <- OStatus.find_make_or_update_user(doc),
+    with {:ok, actor} <- OStatus.find_make_or_update_actor(doc),
          id when not is_nil(id) <- XML.string_from_xpath("/entry/id", entry),
          followed_uri when not is_nil(followed_uri) <-
            XML.string_from_xpath("/entry/activity:object/id", entry),
          {:ok, followed} <- OStatus.find_or_make_user(followed_uri),
+         {:locked, false} <- {:locked, followed.info.locked},
          {:ok, activity} <- ActivityPub.follow(actor, followed, id, false) do
       User.follow(actor, followed)
       {:ok, activity}
+    else
+      {:locked, true} ->
+        {:error, "It's not possible to follow locked accounts over OStatus"}
     end
   end
 end

lib/pleroma/web/ostatus/handlers/note_handler.ex

@@ -4,6 +4,7 @@
 
 defmodule Pleroma.Web.OStatus.NoteHandler do
   require Logger
+  require Pleroma.Constants
 
   alias Pleroma.Activity
   alias Pleroma.Object
@@ -49,7 +50,7 @@ defmodule Pleroma.Web.OStatus.NoteHandler do
   def get_collection_mentions(entry) do
     transmogrify = fn
       "http://activityschema.org/collection/public" ->
-        "https://www.w3.org/ns/activitystreams#Public"
+        Pleroma.Constants.as_public()
 
       group ->
         group
@@ -110,7 +111,7 @@ defmodule Pleroma.Web.OStatus.NoteHandler do
     with id <- XML.string_from_xpath("//id", entry),
          activity when is_nil(activity) <- Activity.get_create_by_object_ap_id_with_object(id),
          [author] <- :xmerl_xpath.string('//author[1]', doc),
-         {:ok, actor} <- OStatus.find_make_or_update_user(author),
+         {:ok, actor} <- OStatus.find_make_or_update_actor(author),
          content_html <- OStatus.get_content(entry),
          cw <- OStatus.get_cw(entry),
          in_reply_to <- XML.string_from_xpath("//thr:in-reply-to[1]/@ref", entry),
@@ -126,7 +127,7 @@ defmodule Pleroma.Web.OStatus.NoteHandler do
          to <- make_to_list(actor, mentions),
          date <- XML.string_from_xpath("//published", entry),
          unlisted <- XML.string_from_xpath("//mastodon:scope", entry) == "unlisted",
-         cc <- if(unlisted, do: ["https://www.w3.org/ns/activitystreams#Public"], else: []),
+         cc <- if(unlisted, do: [Pleroma.Constants.as_public()], else: []),
          note <-
            CommonAPI.Utils.make_note_data(
              actor.ap_id,

lib/pleroma/web/ostatus/handlers/unfollow_handler.ex

@@ -9,7 +9,7 @@ defmodule Pleroma.Web.OStatus.UnfollowHandler do
   alias Pleroma.Web.XML
 
   def handle(entry, doc) do
-    with {:ok, actor} <- OStatus.find_make_or_update_user(doc),
+    with {:ok, actor} <- OStatus.find_make_or_update_actor(doc),
          id when not is_nil(id) <- XML.string_from_xpath("/entry/id", entry),
          followed_uri when not is_nil(followed_uri) <-
            XML.string_from_xpath("/entry/activity:object/id", entry),

lib/pleroma/web/ostatus/ostatus.ex

@@ -56,7 +56,7 @@ defmodule Pleroma.Web.OStatus do
 
   def handle_incoming(xml_string, options \\ []) do
     with doc when doc != :error <- parse_document(xml_string) do
-      with {:ok, actor_user} <- find_make_or_update_user(doc),
+      with {:ok, actor_user} <- find_make_or_update_actor(doc),
            do: Pleroma.Instances.set_reachable(actor_user.ap_id)
 
       entries = :xmerl_xpath.string('//entry', doc)
@@ -120,7 +120,7 @@ defmodule Pleroma.Web.OStatus do
   end
 
   def make_share(entry, doc, retweeted_activity) do
-    with {:ok, actor} <- find_make_or_update_user(doc),
+    with {:ok, actor} <- find_make_or_update_actor(doc),
          %Object{} = object <- Object.normalize(retweeted_activity),
          id when not is_nil(id) <- string_from_xpath("/entry/id", entry),
          {:ok, activity, _object} = ActivityPub.announce(actor, object, id, false) do
@@ -138,7 +138,7 @@ defmodule Pleroma.Web.OStatus do
   end
 
   def make_favorite(entry, doc, favorited_activity) do
-    with {:ok, actor} <- find_make_or_update_user(doc),
+    with {:ok, actor} <- find_make_or_update_actor(doc),
          %Object{} = object <- Object.normalize(favorited_activity),
          id when not is_nil(id) <- string_from_xpath("/entry/id", entry),
          {:ok, activity, _object} = ActivityPub.like(actor, object, id, false) do
@@ -264,11 +264,18 @@ defmodule Pleroma.Web.OStatus do
     end
   end
 
-  def find_make_or_update_user(doc) do
+  def find_make_or_update_actor(doc) do
     uri = string_from_xpath("//author/uri[1]", doc)
 
-    with {:ok, user} <- find_or_make_user(uri) do
+    with {:ok, %User{} = user} <- find_or_make_user(uri),
+         {:ap_enabled, false} <- {:ap_enabled, User.ap_enabled?(user)} do
       maybe_update(doc, user)
+    else
+      {:ap_enabled, true} ->
+        {:error, :invalid_protocol}
+
+      _ ->
+        {:error, :unknown_user}
     end
   end
 

lib/pleroma/web/ostatus/ostatus_controller.ex

@@ -5,6 +5,7 @@
 defmodule Pleroma.Web.OStatus.OStatusController do
   use Pleroma.Web, :controller
 
+  alias Fallback.RedirectController
   alias Pleroma.Activity
   alias Pleroma.Object
   alias Pleroma.User
@@ -12,42 +13,44 @@ defmodule Pleroma.Web.OStatus.OStatusController do
   alias Pleroma.Web.ActivityPub.ActivityPubController
   alias Pleroma.Web.ActivityPub.ObjectView
   alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.Endpoint
   alias Pleroma.Web.Federator
+  alias Pleroma.Web.Metadata.PlayerView
   alias Pleroma.Web.OStatus
   alias Pleroma.Web.OStatus.ActivityRepresenter
   alias Pleroma.Web.OStatus.FeedRepresenter
+  alias Pleroma.Web.Router
   alias Pleroma.Web.XML
 
   plug(Pleroma.Web.FederatingPlug when action in [:salmon_incoming])
 
+  plug(
+    Pleroma.Plugs.SetFormatPlug
+    when action in [:feed_redirect, :object, :activity, :notice]
+  )
+
   action_fallback(:errors)
 
+  def feed_redirect(%{assigns: %{format: "html"}} = conn, %{"nickname" => nickname}) do
+    with {_, %User{} = user} <-
+           {:fetch_user, User.get_cached_by_nickname_or_id(nickname)} do
+      RedirectController.redirector_with_meta(conn, %{user: user})
+    end
+  end
+
+  def feed_redirect(%{assigns: %{format: format}} = conn, _params)
+      when format in ["json", "activity+json"] do
+    ActivityPubController.call(conn, :user)
+  end
+
   def feed_redirect(conn, %{"nickname" => nickname}) do
-    case get_format(conn) do
-      "html" ->
-        with %User{} = user <- User.get_cached_by_nickname_or_id(nickname) do
-          Fallback.RedirectController.redirector_with_meta(conn, %{user: user})
-        else
-          nil -> {:error, :not_found}
-        end
-
-      "activity+json" ->
-        ActivityPubController.call(conn, :user)
-
-      "json" ->
-        ActivityPubController.call(conn, :user)
-
-      _ ->
-        with %User{} = user <- User.get_cached_by_nickname(nickname) do
-          redirect(conn, external: OStatus.feed_path(user))
-        else
-          nil -> {:error, :not_found}
-        end
+    with {_, %User{} = user} <- {:fetch_user, User.get_cached_by_nickname(nickname)} do
+      redirect(conn, external: OStatus.feed_path(user))
     end
   end
 
   def feed(conn, %{"nickname" => nickname} = params) do
-    with %User{} = user <- User.get_cached_by_nickname(nickname) do
+    with {_, %User{} = user} <- {:fetch_user, User.get_cached_by_nickname(nickname)} do
       query_params =
         Map.take(params, ["max_id"])
         |> Map.merge(%{"whole_db" => true, "actor_id" => user.ap_id})
@@ -65,8 +68,6 @@ defmodule Pleroma.Web.OStatus.OStatusController do
       conn
       |> put_resp_content_type("application/atom+xml")
       |> send_resp(200, response)
-    else
-      nil -> {:error, :not_found}
     end
   end
 
@@ -97,93 +98,82 @@ defmodule Pleroma.Web.OStatus.OStatusController do
     |> send_resp(200, "")
   end
 
-  def object(conn, %{"uuid" => uuid}) do
-    if get_format(conn) in ["activity+json", "json"] do
-      ActivityPubController.call(conn, :object)
-    else
-      with id <- o_status_url(conn, :object, uuid),
-           {_, %Activity{} = activity} <-
-             {:activity, Activity.get_create_by_object_ap_id_with_object(id)},
-           {_, true} <- {:public?, Visibility.is_public?(activity)},
-           %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do
-        case get_format(conn) do
-          "html" -> redirect(conn, to: "/notice/#{activity.id}")
-          _ -> represent_activity(conn, nil, activity, user)
-        end
-      else
-        {:public?, false} ->
-          {:error, :not_found}
+  def object(%{assigns: %{format: format}} = conn, %{"uuid" => _uuid})
+      when format in ["json", "activity+json"] do
+    ActivityPubController.call(conn, :object)
+  end
 
-        {:activity, nil} ->
-          {:error, :not_found}
-
-        e ->
-          e
+  def object(%{assigns: %{format: format}} = conn, %{"uuid" => uuid}) do
+    with id <- o_status_url(conn, :object, uuid),
+         {_, %Activity{} = activity} <-
+           {:activity, Activity.get_create_by_object_ap_id_with_object(id)},
+         {_, true} <- {:public?, Visibility.is_public?(activity)},
+         %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do
+      case format do
+        "html" -> redirect(conn, to: "/notice/#{activity.id}")
+        _ -> represent_activity(conn, nil, activity, user)
       end
+    else
+      reason when reason in [{:public?, false}, {:activity, nil}] ->
+        {:error, :not_found}
+
+      e ->
+        e
     end
   end
 
-  def activity(conn, %{"uuid" => uuid}) do
-    if get_format(conn) in ["activity+json", "json"] do
-      ActivityPubController.call(conn, :activity)
-    else
-      with id <- o_status_url(conn, :activity, uuid),
-           {_, %Activity{} = activity} <- {:activity, Activity.normalize(id)},
-           {_, true} <- {:public?, Visibility.is_public?(activity)},
-           %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do
-        case format = get_format(conn) do
-          "html" -> redirect(conn, to: "/notice/#{activity.id}")
-          _ -> represent_activity(conn, format, activity, user)
-        end
-      else
-        {:public?, false} ->
-          {:error, :not_found}
+  def activity(%{assigns: %{format: format}} = conn, %{"uuid" => _uuid})
+      when format in ["json", "activity+json"] do
+    ActivityPubController.call(conn, :activity)
+  end
 
-        {:activity, nil} ->
-          {:error, :not_found}
-
-        e ->
-          e
+  def activity(%{assigns: %{format: format}} = conn, %{"uuid" => uuid}) do
+    with id <- o_status_url(conn, :activity, uuid),
+         {_, %Activity{} = activity} <- {:activity, Activity.normalize(id)},
+         {_, true} <- {:public?, Visibility.is_public?(activity)},
+         %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do
+      case format do
+        "html" -> redirect(conn, to: "/notice/#{activity.id}")
+        _ -> represent_activity(conn, format, activity, user)
       end
+    else
+      reason when reason in [{:public?, false}, {:activity, nil}] ->
+        {:error, :not_found}
+
+      e ->
+        e
     end
   end
 
-  def notice(conn, %{"id" => id}) do
+  def notice(%{assigns: %{format: format}} = conn, %{"id" => id}) do
     with {_, %Activity{} = activity} <- {:activity, Activity.get_by_id_with_object(id)},
          {_, true} <- {:public?, Visibility.is_public?(activity)},
          %User{} = user <- User.get_cached_by_ap_id(activity.data["actor"]) do
-      case format = get_format(conn) do
-        "html" ->
-          if activity.data["type"] == "Create" do
-            %Object{} = object = Object.normalize(activity)
+      cond do
+        format == "html" && activity.data["type"] == "Create" ->
+          %Object{} = object = Object.normalize(activity)
 
-            Fallback.RedirectController.redirector_with_meta(conn, %{
+          RedirectController.redirector_with_meta(
+            conn,
+            %{
               activity_id: activity.id,
               object: object,
-              url:
-                Pleroma.Web.Router.Helpers.o_status_url(
-                  Pleroma.Web.Endpoint,
-                  :notice,
-                  activity.id
-                ),
+              url: Router.Helpers.o_status_url(Endpoint, :notice, activity.id),
               user: user
-            })
-          else
-            Fallback.RedirectController.redirector(conn, nil)
-          end
+            }
+          )
 
-        _ ->
+        format == "html" ->
+          RedirectController.redirector(conn, nil)
+
+        true ->
           represent_activity(conn, format, activity, user)
       end
     else
-      {:public?, false} ->
+      reason when reason in [{:public?, false}, {:activity, nil}] ->
         conn
         |> put_status(404)
-        |> Fallback.RedirectController.redirector(nil, 404)
-
-      {:activity, nil} ->
-        conn
-        |> Fallback.RedirectController.redirector(nil, 404)
+        |> RedirectController.redirector(nil, 404)
 
       e ->
         e
@@ -204,13 +194,13 @@ defmodule Pleroma.Web.OStatus.OStatusController do
         "content-security-policy",
         "default-src 'none';style-src 'self' 'unsafe-inline';img-src 'self' data: https:; media-src 'self' https:;"
       )
-      |> put_view(Pleroma.Web.Metadata.PlayerView)
+      |> put_view(PlayerView)
       |> render("player.html", url)
     else
       _error ->
         conn
         |> put_status(404)
-        |> Fallback.RedirectController.redirector(nil, 404)
+        |> RedirectController.redirector(nil, 404)
     end
   end
 
@@ -248,6 +238,8 @@ defmodule Pleroma.Web.OStatus.OStatusController do
     render_error(conn, :not_found, "Not found")
   end
 
+  def errors(conn, {:fetch_user, nil}), do: errors(conn, {:error, :not_found})
+
   def errors(conn, _) do
     render_error(conn, :internal_server_error, "Something went wrong")
   end

lib/pleroma/web/rich_media/parser.ex

@@ -24,6 +24,7 @@ defmodule Pleroma.Web.RichMedia.Parser do
         Cachex.fetch!(:rich_media_cache, url, fn _ ->
           {:commit, parse_url(url)}
         end)
+        |> set_ttl_based_on_image(url)
       rescue
         e ->
           {:error, "Cachex error: #{inspect(e)}"}
@@ -31,12 +32,57 @@ defmodule Pleroma.Web.RichMedia.Parser do
     end
   end
 
+  @doc """
+  Set the rich media cache based on the expiration time of image.
+
+  Adopt behaviour `Pleroma.Web.RichMedia.Parser.TTL`
+
+  ## Example
+
+      defmodule MyModule do
+        @behaviour Pleroma.Web.RichMedia.Parser.TTL
+        def ttl(data, url) do
+          image_url = Map.get(data, :image)
+          # do some parsing in the url and get the ttl of the image
+          # and return ttl is unix time
+          parse_ttl_from_url(image_url)
+        end
+      end
+
+  Define the module in the config
+
+      config :pleroma, :rich_media,
+        ttl_setters: [MyModule]
+  """
+  def set_ttl_based_on_image({:ok, data}, url) do
+    with {:ok, nil} <- Cachex.ttl(:rich_media_cache, url),
+         ttl when is_number(ttl) <- get_ttl_from_image(data, url) do
+      Cachex.expire_at(:rich_media_cache, url, ttl * 1000)
+      {:ok, data}
+    else
+      _ ->
+        {:ok, data}
+    end
+  end
+
+  defp get_ttl_from_image(data, url) do
+    Pleroma.Config.get([:rich_media, :ttl_setters])
+    |> Enum.reduce({:ok, nil}, fn
+      module, {:ok, _ttl} ->
+        module.ttl(data, url)
+
+      _, error ->
+        error
+    end)
+  end
+
   defp parse_url(url) do
     try do
       {:ok, %Tesla.Env{body: html}} = Pleroma.HTTP.get(url, [], adapter: @hackney_options)
 
       html
       |> maybe_parse()
+      |> Map.put(:url, url)
       |> clean_parsed_data()
       |> check_parsed_data()
     rescue

lib/pleroma/web/rich_media/parsers/ttl/aws_signed_url.ex

@@ -0,0 +1,51 @@
+defmodule Pleroma.Web.RichMedia.Parser.TTL.AwsSignedUrl do
+  @behaviour Pleroma.Web.RichMedia.Parser.TTL
+
+  @impl Pleroma.Web.RichMedia.Parser.TTL
+  def ttl(data, _url) do
+    image = Map.get(data, :image)
+
+    if is_aws_signed_url(image) do
+      image
+      |> parse_query_params()
+      |> format_query_params()
+      |> get_expiration_timestamp()
+    end
+  end
+
+  defp is_aws_signed_url(""), do: nil
+  defp is_aws_signed_url(nil), do: nil
+
+  defp is_aws_signed_url(image) when is_binary(image) do
+    %URI{host: host, query: query} = URI.parse(image)
+
+    if String.contains?(host, "amazonaws.com") and String.contains?(query, "X-Amz-Expires") do
+      image
+    else
+      nil
+    end
+  end
+
+  defp is_aws_signed_url(_), do: nil
+
+  defp parse_query_params(image) do
+    %URI{query: query} = URI.parse(image)
+    query
+  end
+
+  defp format_query_params(query) do
+    query
+    |> String.split(~r/&|=/)
+    |> Enum.chunk_every(2)
+    |> Map.new(fn [k, v] -> {k, v} end)
+  end
+
+  defp get_expiration_timestamp(params) when is_map(params) do
+    {:ok, date} =
+      params
+      |> Map.get("X-Amz-Date")
+      |> Timex.parse("{ISO:Basic:Z}")
+
+    Timex.to_unix(date) + String.to_integer(Map.get(params, "X-Amz-Expires"))
+  end
+end

lib/pleroma/web/rich_media/parsers/ttl/ttl.ex

@@ -0,0 +1,3 @@
+defmodule Pleroma.Web.RichMedia.Parser.TTL do
+  @callback ttl(Map.t(), String.t()) :: {:ok, Integer.t()} | {:error, String.t()}
+end

lib/pleroma/web/router.ex

@@ -154,22 +154,12 @@ defmodule Pleroma.Web.Router do
     post("/users/follow", AdminAPIController, :user_follow)
     post("/users/unfollow", AdminAPIController, :user_unfollow)
 
-    # TODO: to be removed at version 1.0
-    delete("/user", AdminAPIController, :user_delete)
-    post("/user", AdminAPIController, :user_create)
-
     delete("/users", AdminAPIController, :user_delete)
     post("/users", AdminAPIController, :user_create)
     patch("/users/:nickname/toggle_activation", AdminAPIController, :user_toggle_activation)
     put("/users/tag", AdminAPIController, :tag_users)
     delete("/users/tag", AdminAPIController, :untag_users)
 
-    # TODO: to be removed at version 1.0
-    get("/permission_group/:nickname", AdminAPIController, :right_get)
-    get("/permission_group/:nickname/:permission_group", AdminAPIController, :right_get)
-    post("/permission_group/:nickname/:permission_group", AdminAPIController, :right_add)
-    delete("/permission_group/:nickname/:permission_group", AdminAPIController, :right_delete)
-
     get("/users/:nickname/permission_group", AdminAPIController, :right_get)
     get("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_get)
     post("/users/:nickname/permission_group/:permission_group", AdminAPIController, :right_add)
@@ -190,13 +180,11 @@ defmodule Pleroma.Web.Router do
     post("/users/revoke_invite", AdminAPIController, :revoke_invite)
     post("/users/email_invite", AdminAPIController, :email_invite)
 
-    # TODO: to be removed at version 1.0
-    get("/password_reset", AdminAPIController, :get_password_reset)
-
     get("/users/:nickname/password_reset", AdminAPIController, :get_password_reset)
 
     get("/users", AdminAPIController, :list_users)
     get("/users/:nickname", AdminAPIController, :user_show)
+    get("/users/:nickname/statuses", AdminAPIController, :list_user_statuses)
 
     get("/reports", AdminAPIController, :list_reports)
     get("/reports/:id", AdminAPIController, :report_show)
@@ -208,6 +196,8 @@ defmodule Pleroma.Web.Router do
 
     get("/config", AdminAPIController, :config_show)
     post("/config", AdminAPIController, :config_update)
+    get("/config/migrate_to_db", AdminAPIController, :migrate_to_db)
+    get("/config/migrate_from_db", AdminAPIController, :migrate_from_db)
   end
 
   scope "/", Pleroma.Web.TwitterAPI do
@@ -424,6 +414,12 @@ defmodule Pleroma.Web.Router do
 
     get("/accounts/search", SearchController, :account_search)
 
+    post(
+      "/pleroma/accounts/confirmation_resend",
+      MastodonAPIController,
+      :account_confirmation_resend
+    )
+
     scope [] do
       pipe_through(:oauth_read_or_public)
 
@@ -617,6 +613,7 @@ defmodule Pleroma.Web.Router do
   pipeline :activitypub do
     plug(:accepts, ["activity+json", "json"])
     plug(Pleroma.Web.Plugs.HTTPSignaturePlug)
+    plug(Pleroma.Web.Plugs.MappedSignatureToIdentityPlug)
   end
 
   scope "/", Pleroma.Web.ActivityPub do
@@ -662,6 +659,12 @@ defmodule Pleroma.Web.Router do
     end
   end
 
+  scope "/", Pleroma.Web.ActivityPub do
+    pipe_through(:activitypub)
+    post("/inbox", ActivityPubController, :inbox)
+    post("/users/:nickname/inbox", ActivityPubController, :inbox)
+  end
+
   scope "/relay", Pleroma.Web.ActivityPub do
     pipe_through(:ap_service_actor)
 
@@ -676,12 +679,6 @@ defmodule Pleroma.Web.Router do
     post("/inbox", ActivityPubController, :inbox)
   end
 
-  scope "/", Pleroma.Web.ActivityPub do
-    pipe_through(:activitypub)
-    post("/inbox", ActivityPubController, :inbox)
-    post("/users/:nickname/inbox", ActivityPubController, :inbox)
-  end
-
   scope "/.well-known", Pleroma.Web do
     pipe_through(:well_known)
 
@@ -703,7 +700,7 @@ defmodule Pleroma.Web.Router do
     post("/auth/password", MastodonAPIController, :password_reset)
 
     scope [] do
-      pipe_through(:oauth_read_or_public)
+      pipe_through(:oauth_read)
       get("/web/*path", MastodonAPIController, :index)
     end
   end
@@ -740,68 +737,3 @@ defmodule Pleroma.Web.Router do
     options("/*path", RedirectController, :empty)
   end
 end
-
-defmodule Fallback.RedirectController do
-  use Pleroma.Web, :controller
-  require Logger
-  alias Pleroma.User
-  alias Pleroma.Web.Metadata
-
-  def api_not_implemented(conn, _params) do
-    conn
-    |> put_status(404)
-    |> json(%{error: "Not implemented"})
-  end
-
-  def redirector(conn, _params, code \\ 200) do
-    conn
-    |> put_resp_content_type("text/html")
-    |> send_file(code, index_file_path())
-  end
-
-  def redirector_with_meta(conn, %{"maybe_nickname_or_id" => maybe_nickname_or_id} = params) do
-    with %User{} = user <- User.get_cached_by_nickname_or_id(maybe_nickname_or_id) do
-      redirector_with_meta(conn, %{user: user})
-    else
-      nil ->
-        redirector(conn, params)
-    end
-  end
-
-  def redirector_with_meta(conn, params) do
-    {:ok, index_content} = File.read(index_file_path())
-
-    tags =
-      try do
-        Metadata.build_tags(params)
-      rescue
-        e ->
-          Logger.error(
-            "Metadata rendering for #{conn.request_path} failed.\n" <>
-              Exception.format(:error, e, __STACKTRACE__)
-          )
-
-          ""
-      end
-
-    response = String.replace(index_content, "<!--server-generated-meta-->", tags)
-
-    conn
-    |> put_resp_content_type("text/html")
-    |> send_resp(200, response)
-  end
-
-  def index_file_path do
-    Pleroma.Plugs.InstanceStatic.file_path("index.html")
-  end
-
-  def registration_page(conn, params) do
-    redirector(conn, params)
-  end
-
-  def empty(conn, _params) do
-    conn
-    |> put_status(204)
-    |> text("")
-  end
-end

lib/pleroma/web/streamer.ex

@@ -13,6 +13,7 @@ defmodule Pleroma.Web.Streamer do
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.ActivityPub.Visibility
+  alias Pleroma.Web.CommonAPI
   alias Pleroma.Web.MastodonAPI.NotificationView
 
   @keepalive_interval :timer.seconds(30)
@@ -118,10 +119,14 @@ defmodule Pleroma.Web.Streamer do
     topics
     |> Map.get("#{topic}:#{item.user_id}", [])
     |> Enum.each(fn socket ->
-      send(
-        socket.transport_pid,
-        {:text, represent_notification(socket.assigns[:user], item)}
-      )
+      with %User{} = user <- User.get_cached_by_ap_id(socket.assigns[:user].ap_id),
+           true <- should_send?(user, item),
+           false <- CommonAPI.thread_muted?(user, item.activity) do
+        send(
+          socket.transport_pid,
+          {:text, represent_notification(socket.assigns[:user], item)}
+        )
+      end
     end)
 
     {:noreply, topics}
@@ -225,19 +230,37 @@ defmodule Pleroma.Web.Streamer do
     |> Jason.encode!()
   end
 
+  defp should_send?(%User{} = user, %Activity{} = item) do
+    blocks = user.info.blocks || []
+    mutes = user.info.mutes || []
+    reblog_mutes = user.info.muted_reblogs || []
+    domain_blocks = Pleroma.Web.ActivityPub.MRF.subdomains_regex(user.info.domain_blocks)
+
+    with parent when not is_nil(parent) <- Object.normalize(item),
+         true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)),
+         true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)),
+         %{host: item_host} <- URI.parse(item.actor),
+         %{host: parent_host} <- URI.parse(parent.data["actor"]),
+         false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, item_host),
+         false <- Pleroma.Web.ActivityPub.MRF.subdomain_match?(domain_blocks, parent_host),
+         true <- thread_containment(item, user) do
+      true
+    else
+      _ -> false
+    end
+  end
+
+  defp should_send?(%User{} = user, %Notification{activity: activity}) do
+    should_send?(user, activity)
+  end
+
   def push_to_socket(topics, topic, %Activity{data: %{"type" => "Announce"}} = item) do
     Enum.each(topics[topic] || [], fn socket ->
       # Get the current user so we have up-to-date blocks etc.
       if socket.assigns[:user] do
         user = User.get_cached_by_ap_id(socket.assigns[:user].ap_id)
-        blocks = user.info.blocks || []
-        mutes = user.info.mutes || []
-        reblog_mutes = user.info.muted_reblogs || []
 
-        with parent when not is_nil(parent) <- Object.normalize(item),
-             true <- Enum.all?([blocks, mutes, reblog_mutes], &(item.actor not in &1)),
-             true <- Enum.all?([blocks, mutes], &(parent.data["actor"] not in &1)),
-             true <- thread_containment(item, user) do
+        if should_send?(user, item) do
           send(socket.transport_pid, {:text, represent_update(item, user)})
         end
       else

lib/pleroma/web/twitter_api/controllers/util_controller.ex

@@ -8,22 +8,25 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
   require Logger
 
   alias Pleroma.Activity
+  alias Pleroma.Config
   alias Pleroma.Emoji
+  alias Pleroma.Healthcheck
   alias Pleroma.Notification
   alias Pleroma.Plugs.AuthenticationPlug
   alias Pleroma.User
   alias Pleroma.Web
-  alias Pleroma.Web.ActivityPub.ActivityPub
   alias Pleroma.Web.CommonAPI
-  alias Pleroma.Web.OStatus
   alias Pleroma.Web.WebFinger
 
+  plug(Pleroma.Plugs.SetFormatPlug when action in [:config, :version])
+
   def help_test(conn, _params) do
     json(conn, "ok")
   end
 
   def remote_subscribe(conn, %{"nickname" => nick, "profile" => _}) do
-    with %User{} = user <- User.get_cached_by_nickname(nick), avatar = User.avatar_url(user) do
+    with %User{} = user <- User.get_cached_by_nickname(nick),
+         avatar = User.avatar_url(user) do
       conn
       |> render("subscribe.html", %{nickname: nick, avatar: avatar, error: false})
     else
@@ -57,27 +60,25 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
       %Activity{id: activity_id} = Activity.get_create_by_object_ap_id(object.data["id"])
       redirect(conn, to: "/notice/#{activity_id}")
     else
-      {err, followee} = OStatus.find_or_make_user(acct)
-      avatar = User.avatar_url(followee)
-      name = followee.nickname
-      id = followee.id
-
-      if !!user do
+      with {:ok, followee} <- User.get_or_fetch(acct) do
         conn
-        |> render("follow.html", %{error: err, acct: acct, avatar: avatar, name: name, id: id})
-      else
-        conn
-        |> render("follow_login.html", %{
+        |> render(follow_template(user), %{
           error: false,
           acct: acct,
-          avatar: avatar,
-          name: name,
-          id: id
+          avatar: User.avatar_url(followee),
+          name: followee.nickname,
+          id: followee.id
         })
+      else
+        {:error, _reason} ->
+          render(conn, follow_template(user), %{error: :error})
       end
     end
   end
 
+  defp follow_template(%User{} = _user), do: "follow.html"
+  defp follow_template(_), do: "follow_login.html"
+
   defp is_status?(acct) do
     case Pleroma.Object.Fetcher.fetch_and_contain_remote_object_from_id(acct) do
       {:ok, %{"type" => type}} when type in ["Article", "Note", "Video", "Page", "Question"] ->
@@ -91,50 +92,53 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
   def do_remote_follow(conn, %{
         "authorization" => %{"name" => username, "password" => password, "id" => id}
       }) do
-    followee = User.get_cached_by_id(id)
-    avatar = User.avatar_url(followee)
-    name = followee.nickname
-
-    with %User{} = user <- User.get_cached_by_nickname(username),
-         true <- AuthenticationPlug.checkpw(password, user.password_hash),
-         %User{} = _followed <- User.get_cached_by_id(id),
-         {:ok, follower} <- User.follow(user, followee),
-         {:ok, _activity} <- ActivityPub.follow(follower, followee) do
+    with %User{} = followee <- User.get_cached_by_id(id),
+         {_, %User{} = user, _} <- {:auth, User.get_cached_by_nickname(username), followee},
+         {_, true, _} <- {
+           :auth,
+           AuthenticationPlug.checkpw(password, user.password_hash),
+           followee
+         },
+         {:ok, _follower, _followee, _activity} <- CommonAPI.follow(user, followee) do
       conn
       |> render("followed.html", %{error: false})
     else
       # Was already following user
       {:error, "Could not follow user:" <> _rest} ->
-        render(conn, "followed.html", %{error: false})
+        render(conn, "followed.html", %{error: "Error following account"})
 
-      _e ->
+      {:auth, _, followee} ->
         conn
         |> render("follow_login.html", %{
           error: "Wrong username or password",
           id: id,
-          name: name,
-          avatar: avatar
+          name: followee.nickname,
+          avatar: User.avatar_url(followee)
         })
+
+      e ->
+        Logger.debug("Remote follow failed with error #{inspect(e)}")
+        render(conn, "followed.html", %{error: "Something went wrong."})
     end
   end
 
   def do_remote_follow(%{assigns: %{user: user}} = conn, %{"user" => %{"id" => id}}) do
-    with %User{} = followee <- User.get_cached_by_id(id),
-         {:ok, follower} <- User.follow(user, followee),
-         {:ok, _activity} <- ActivityPub.follow(follower, followee) do
+    with {:fetch_user, %User{} = followee} <- {:fetch_user, User.get_cached_by_id(id)},
+         {:ok, _follower, _followee, _activity} <- CommonAPI.follow(user, followee) do
       conn
       |> render("followed.html", %{error: false})
     else
       # Was already following user
       {:error, "Could not follow user:" <> _rest} ->
-        conn
-        |> render("followed.html", %{error: false})
+        render(conn, "followed.html", %{error: "Error following account"})
+
+      {:fetch_user, error} ->
+        Logger.debug("Remote follow failed with error #{inspect(error)}")
+        render(conn, "followed.html", %{error: "Could not find user"})
 
       e ->
         Logger.debug("Remote follow failed with error #{inspect(e)}")
-
-        conn
-        |> render("followed.html", %{error: inspect(e)})
+        render(conn, "followed.html", %{error: "Something went wrong."})
     end
   end
 
@@ -149,67 +153,70 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
     end
   end
 
+  def config(%{assigns: %{format: "xml"}} = conn, _params) do
+    instance = Pleroma.Config.get(:instance)
+
+    response = """
+    <config>
+    <site>
+    <name>#{Keyword.get(instance, :name)}</name>
+    <site>#{Web.base_url()}</site>
+    <textlimit>#{Keyword.get(instance, :limit)}</textlimit>
+    <closed>#{!Keyword.get(instance, :registrations_open)}</closed>
+    </site>
+    </config>
+    """
+
+    conn
+    |> put_resp_content_type("application/xml")
+    |> send_resp(200, response)
+  end
+
   def config(conn, _params) do
     instance = Pleroma.Config.get(:instance)
 
-    case get_format(conn) do
-      "xml" ->
-        response = """
-        <config>
-          <site>
-            <name>#{Keyword.get(instance, :name)}</name>
-            <site>#{Web.base_url()}</site>
-            <textlimit>#{Keyword.get(instance, :limit)}</textlimit>
-            <closed>#{!Keyword.get(instance, :registrations_open)}</closed>
-          </site>
-        </config>
-        """
+    vapid_public_key = Keyword.get(Pleroma.Web.Push.vapid_config(), :public_key)
 
-        conn
-        |> put_resp_content_type("application/xml")
-        |> send_resp(200, response)
+    uploadlimit = %{
+      uploadlimit: to_string(Keyword.get(instance, :upload_limit)),
+      avatarlimit: to_string(Keyword.get(instance, :avatar_upload_limit)),
+      backgroundlimit: to_string(Keyword.get(instance, :background_upload_limit)),
+      bannerlimit: to_string(Keyword.get(instance, :banner_upload_limit))
+    }
 
-      _ ->
-        vapid_public_key = Keyword.get(Pleroma.Web.Push.vapid_config(), :public_key)
+    data = %{
+      name: Keyword.get(instance, :name),
+      description: Keyword.get(instance, :description),
+      server: Web.base_url(),
+      textlimit: to_string(Keyword.get(instance, :limit)),
+      uploadlimit: uploadlimit,
+      closed: bool_to_val(Keyword.get(instance, :registrations_open), "0", "1"),
+      private: bool_to_val(Keyword.get(instance, :public, true), "0", "1"),
+      vapidPublicKey: vapid_public_key,
+      accountActivationRequired:
+        bool_to_val(Keyword.get(instance, :account_activation_required, false)),
+      invitesEnabled: bool_to_val(Keyword.get(instance, :invites_enabled, false)),
+      safeDMMentionsEnabled: bool_to_val(Pleroma.Config.get([:instance, :safe_dm_mentions]))
+    }
 
-        uploadlimit = %{
-          uploadlimit: to_string(Keyword.get(instance, :upload_limit)),
-          avatarlimit: to_string(Keyword.get(instance, :avatar_upload_limit)),
-          backgroundlimit: to_string(Keyword.get(instance, :background_upload_limit)),
-          bannerlimit: to_string(Keyword.get(instance, :banner_upload_limit))
-        }
-
-        data = %{
-          name: Keyword.get(instance, :name),
-          description: Keyword.get(instance, :description),
-          server: Web.base_url(),
-          textlimit: to_string(Keyword.get(instance, :limit)),
-          uploadlimit: uploadlimit,
-          closed: if(Keyword.get(instance, :registrations_open), do: "0", else: "1"),
-          private: if(Keyword.get(instance, :public, true), do: "0", else: "1"),
-          vapidPublicKey: vapid_public_key,
-          accountActivationRequired:
-            if(Keyword.get(instance, :account_activation_required, false), do: "1", else: "0"),
-          invitesEnabled: if(Keyword.get(instance, :invites_enabled, false), do: "1", else: "0"),
-          safeDMMentionsEnabled:
-            if(Pleroma.Config.get([:instance, :safe_dm_mentions]), do: "1", else: "0")
-        }
+    managed_config = Keyword.get(instance, :managed_config)
 
+    data =
+      if managed_config do
         pleroma_fe = Pleroma.Config.get([:frontend_configurations, :pleroma_fe])
+        Map.put(data, "pleromafe", pleroma_fe)
+      else
+        data
+      end
 
-        managed_config = Keyword.get(instance, :managed_config)
-
-        data =
-          if managed_config do
-            data |> Map.put("pleromafe", pleroma_fe)
-          else
-            data
-          end
-
-        json(conn, %{site: data})
-    end
+    json(conn, %{site: data})
   end
 
+  defp bool_to_val(true), do: "1"
+  defp bool_to_val(_), do: "0"
+  defp bool_to_val(true, val, _), do: val
+  defp bool_to_val(_, _, val), do: val
+
   def frontend_configurations(conn, _params) do
     config =
       Pleroma.Config.get(:frontend_configurations, %{})
@@ -218,20 +225,16 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
     json(conn, config)
   end
 
-  def version(conn, _params) do
+  def version(%{assigns: %{format: "xml"}} = conn, _params) do
     version = Pleroma.Application.named_version()
 
-    case get_format(conn) do
-      "xml" ->
-        response = "<version>#{version}</version>"
+    conn
+    |> put_resp_content_type("application/xml")
+    |> send_resp(200, "<version>#{version}</version>")
+  end
 
-        conn
-        |> put_resp_content_type("application/xml")
-        |> send_resp(200, response)
-
-      _ ->
-        json(conn, version)
-    end
+  def version(conn, _params) do
+    json(conn, Pleroma.Application.named_version())
   end
 
   def emoji(conn, _params) do
@@ -338,20 +341,21 @@ defmodule Pleroma.Web.TwitterAPI.UtilController do
   end
 
   def healthcheck(conn, _params) do
-    info =
-      if Pleroma.Config.get([:instance, :healthcheck]) do
-        Pleroma.Healthcheck.system_info()
-      else
-        %{}
-      end
+    with true <- Config.get([:instance, :healthcheck]),
+         %{healthy: true} = info <- Healthcheck.system_info() do
+      json(conn, info)
+    else
+      %{healthy: false} = info ->
+        service_unavailable(conn, info)
 
-    conn =
-      if info[:healthy] do
-        conn
-      else
-        Plug.Conn.put_status(conn, :service_unavailable)
-      end
+      _ ->
+        service_unavailable(conn, %{})
+    end
+  end
 
-    json(conn, info)
+  defp service_unavailable(conn, info) do
+    conn
+    |> put_status(:service_unavailable)
+    |> json(info)
   end
 end

lib/pleroma/web/twitter_api/twitter_api.ex

@@ -15,6 +15,8 @@ defmodule Pleroma.Web.TwitterAPI.TwitterAPI do
 
   import Ecto.Query
 
+  require Pleroma.Constants
+
   def create_status(%User{} = user, %{"status" => _} = data) do
     CommonAPI.post(user, data)
   end
@@ -286,7 +288,7 @@ defmodule Pleroma.Web.TwitterAPI.TwitterAPI do
       from(
         [a, o] in Activity.with_preloaded_object(Activity),
         where: fragment("?->>'type' = 'Create'", a.data),
-        where: "https://www.w3.org/ns/activitystreams#Public" in a.recipients,
+        where: ^Pleroma.Constants.as_public() in a.recipients,
         where:
           fragment(
             "to_tsvector('english', ?->>'content') @@ plainto_tsquery('english', ?)",

lib/pleroma/web/twitter_api/views/activity_view.ex

@@ -19,6 +19,7 @@ defmodule Pleroma.Web.TwitterAPI.ActivityView do
 
   import Ecto.Query
   require Logger
+  require Pleroma.Constants
 
   defp query_context_ids([]), do: []
 
@@ -91,7 +92,7 @@ defmodule Pleroma.Web.TwitterAPI.ActivityView do
       String.ends_with?(ap_id, "/followers") ->
         nil
 
-      ap_id == "https://www.w3.org/ns/activitystreams#Public" ->
+      ap_id == Pleroma.Constants.as_public() ->
         nil
 
       user = User.get_cached_by_ap_id(ap_id) ->

lib/pleroma/web/twitter_api/views/notification_view.ex

@@ -10,6 +10,8 @@ defmodule Pleroma.Web.TwitterAPI.NotificationView do
   alias Pleroma.Web.TwitterAPI.ActivityView
   alias Pleroma.Web.TwitterAPI.UserView
 
+  require Pleroma.Constants
+
   defp get_user(ap_id, opts) do
     cond do
       user = opts[:users][ap_id] ->
@@ -18,7 +20,7 @@ defmodule Pleroma.Web.TwitterAPI.NotificationView do
       String.ends_with?(ap_id, "/followers") ->
         nil
 
-      ap_id == "https://www.w3.org/ns/activitystreams#Public" ->
+      ap_id == Pleroma.Constants.as_public() ->
         nil
 
       true ->

lib/pleroma/web/web_finger/web_finger.ex

@@ -86,11 +86,17 @@ defmodule Pleroma.Web.WebFinger do
     |> XmlBuilder.to_doc()
   end
 
-  defp get_magic_key(magic_key) do
-    "data:application/magic-public-key," <> magic_key = magic_key
+  defp get_magic_key("data:application/magic-public-key," <> magic_key) do
     {:ok, magic_key}
-  rescue
-    MatchError -> {:error, "Missing magic key data."}
+  end
+
+  defp get_magic_key(nil) do
+    Logger.debug("Undefined magic key.")
+    {:ok, nil}
+  end
+
+  defp get_magic_key(_) do
+    {:error, "Missing magic key data."}
   end
 
   defp webfinger_from_xml(doc) do
@@ -187,6 +193,7 @@ defmodule Pleroma.Web.WebFinger do
     end
   end
 
+  @spec finger(String.t()) :: {:ok, map()} | {:error, any()}
   def finger(account) do
     account = String.trim_leading(account, "@")
 
@@ -220,8 +227,6 @@ defmodule Pleroma.Web.WebFinger do
       else
         with {:ok, doc} <- Jason.decode(body) do
           webfinger_from_json(doc)
-        else
-          {:error, e} -> e
         end
       end
     else

lib/pleroma/web/web_finger/web_finger_controller.ex

@@ -7,6 +7,7 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do
 
   alias Pleroma.Web.WebFinger
 
+  plug(Pleroma.Plugs.SetFormatPlug)
   plug(Pleroma.Web.FederatingPlug)
 
   def host_meta(conn, _params) do
@@ -17,30 +18,28 @@ defmodule Pleroma.Web.WebFinger.WebFingerController do
     |> send_resp(200, xml)
   end
 
-  def webfinger(conn, %{"resource" => resource}) do
-    case get_format(conn) do
-      n when n in ["xml", "xrd+xml"] ->
-        with {:ok, response} <- WebFinger.webfinger(resource, "XML") do
-          conn
-          |> put_resp_content_type("application/xrd+xml")
-          |> send_resp(200, response)
-        else
-          _e -> send_resp(conn, 404, "Couldn't find user")
-        end
-
-      n when n in ["json", "jrd+json"] ->
-        with {:ok, response} <- WebFinger.webfinger(resource, "JSON") do
-          json(conn, response)
-        else
-          _e -> send_resp(conn, 404, "Couldn't find user")
-        end
-
-      _ ->
-        send_resp(conn, 404, "Unsupported format")
+  def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource})
+      when format in ["xml", "xrd+xml"] do
+    with {:ok, response} <- WebFinger.webfinger(resource, "XML") do
+      conn
+      |> put_resp_content_type("application/xrd+xml")
+      |> send_resp(200, response)
+    else
+      _e -> send_resp(conn, 404, "Couldn't find user")
     end
   end
 
-  def webfinger(conn, _params) do
-    send_resp(conn, 400, "Bad Request")
+  def webfinger(%{assigns: %{format: format}} = conn, %{"resource" => resource})
+      when format in ["json", "jrd+json"] do
+    with {:ok, response} <- WebFinger.webfinger(resource, "JSON") do
+      json(conn, response)
+    else
+      _e ->
+        conn
+        |> put_status(404)
+        |> json("Couldn't find user")
+    end
   end
+
+  def webfinger(conn, _params), do: send_resp(conn, 400, "Bad Request")
 end

mix.exs

@@ -138,12 +138,12 @@ defmodule Pleroma.Mixfile do
        ref: "95e8188490e97505c56636c1379ffdf036c1fdde"},
       {:http_signatures,
        git: "https://git.pleroma.social/pleroma/http_signatures.git",
-       ref: "a2a5982fa167fb1352fbd518ce6b606ba233a989"},
+       ref: "293d77bb6f4a67ac8bde1428735c3b42f22cbb30"},
       {:pleroma_job_queue, "~> 0.2.0"},
       {:telemetry, "~> 0.3"},
       {:prometheus_ex, "~> 3.0"},
       {:prometheus_plugs, "~> 1.1"},
-      {:prometheus_phoenix, "~> 1.2"},
+      {:prometheus_phoenix, "~> 1.3"},
       {:prometheus_ecto, "~> 1.4"},
       {:recon, github: "ferd/recon", tag: "2.4.0"},
       {:quack, "~> 0.1.1"},
@@ -151,6 +151,7 @@ defmodule Pleroma.Mixfile do
       {:esshd, "~> 0.1.0", runtime: Application.get_env(:esshd, :enabled, false)},
       {:ex_rated, "~> 1.3"},
       {:junit_formatter, "~> 3.0", only: [:test]},
+      {:ex_const, "~> 0.2"},
       {:plug_static_index_html, "~> 1.0.0"},
       {:excoveralls, "~> 0.11.1", only: :test},
       {:mox, "~> 0.5", only: :test}
@@ -190,12 +191,13 @@ defmodule Pleroma.Mixfile do
            tag = String.trim(tag),
            {describe, 0} <- System.cmd("git", ["describe", "--tags", "--abbrev=8"]),
            describe = String.trim(describe),
-           ahead <- String.replace(describe, tag, "") do
+           ahead <- String.replace(describe, tag, ""),
+           ahead <- String.trim_leading(ahead, "-") do
         {String.replace_prefix(tag, "v", ""), if(ahead != "", do: String.trim(ahead))}
       else
         _ ->
           {commit_hash, 0} = System.cmd("git", ["rev-parse", "--short", "HEAD"])
-          {nil, "-0-g" <> String.trim(commit_hash)}
+          {nil, "0-g" <> String.trim(commit_hash)}
       end
 
     if git_tag && version != git_tag do
@@ -207,14 +209,15 @@ defmodule Pleroma.Mixfile do
     # Branch name as pre-release version component, denoted with a dot
     branch_name =
       with {branch_name, 0} <- System.cmd("git", ["rev-parse", "--abbrev-ref", "HEAD"]),
+           branch_name <- String.trim(branch_name),
            branch_name <- System.get_env("PLEROMA_BUILD_BRANCH") || branch_name,
-           true <- branch_name != "master" do
+           true <- branch_name not in ["master", "HEAD"] do
         branch_name =
           branch_name
           |> String.trim()
           |> String.replace(identifier_filter, "-")
 
-        "." <> branch_name
+        branch_name
       end
 
     build_name =
@@ -234,6 +237,17 @@ defmodule Pleroma.Mixfile do
         env_override -> env_override
       end
 
+    # Pre-release version, denoted by appending a hyphen
+    # and a series of dot separated identifiers
+    pre_release =
+      [git_pre_release, branch_name]
+      |> Enum.filter(fn string -> string && string != "" end)
+      |> Enum.join(".")
+      |> (fn
+            "" -> nil
+            string -> "-" <> String.replace(string, identifier_filter, "-")
+          end).()
+
     # Build metadata, denoted with a plus sign
     build_metadata =
       [build_name, env_name]
@@ -244,7 +258,7 @@ defmodule Pleroma.Mixfile do
             string -> "+" <> String.replace(string, identifier_filter, "-")
           end).()
 
-    [version, git_pre_release, branch_name, build_metadata]
+    [version, pre_release, build_metadata]
     |> Enum.filter(fn string -> string && string != "" end)
     |> Enum.join()
   end

mix.lock

@@ -27,6 +27,7 @@
   "ex2ms": {:hex, :ex2ms, "1.5.0", "19e27f9212be9a96093fed8cdfbef0a2b56c21237196d26760f11dfcfae58e97", [:mix], [], "hexpm"},
   "ex_aws": {:hex, :ex_aws, "2.1.0", "b92651527d6c09c479f9013caa9c7331f19cba38a650590d82ebf2c6c16a1d8a", [:mix], [{:configparser_ex, "~> 2.0", [hex: :configparser_ex, repo: "hexpm", optional: true]}, {:hackney, "1.6.3 or 1.6.5 or 1.7.1 or 1.8.6 or ~> 1.9", [hex: :hackney, repo: "hexpm", optional: true]}, {:jsx, "~> 2.8", [hex: :jsx, repo: "hexpm", optional: true]}, {:poison, ">= 1.2.0", [hex: :poison, repo: "hexpm", optional: true]}, {:sweet_xml, "~> 0.6", [hex: :sweet_xml, repo: "hexpm", optional: true]}, {:xml_builder, "~> 0.1.0", [hex: :xml_builder, repo: "hexpm", optional: true]}], "hexpm"},
   "ex_aws_s3": {:hex, :ex_aws_s3, "2.0.1", "9e09366e77f25d3d88c5393824e613344631be8db0d1839faca49686e99b6704", [:mix], [{:ex_aws, "~> 2.0", [hex: :ex_aws, repo: "hexpm", optional: false]}, {:sweet_xml, ">= 0.0.0", [hex: :sweet_xml, repo: "hexpm", optional: true]}], "hexpm"},
+  "ex_const": {:hex, :ex_const, "0.2.4", "d06e540c9d834865b012a17407761455efa71d0ce91e5831e86881b9c9d82448", [:mix], [], "hexpm"},
   "ex_doc": {:hex, :ex_doc, "0.20.2", "1bd0dfb0304bade58beb77f20f21ee3558cc3c753743ae0ddbb0fd7ba2912331", [:mix], [{:earmark, "~> 1.3", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.10", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"},
   "ex_machina": {:hex, :ex_machina, "2.3.0", "92a5ad0a8b10ea6314b876a99c8c9e3f25f4dde71a2a835845b136b9adaf199a", [:mix], [{:ecto, "~> 2.2 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: true]}, {:ecto_sql, "~> 3.0", [hex: :ecto_sql, repo: "hexpm", optional: true]}], "hexpm"},
   "ex_rated": {:hex, :ex_rated, "1.3.3", "30ecbdabe91f7eaa9d37fa4e81c85ba420f371babeb9d1910adbcd79ec798d27", [:mix], [{:ex2ms, "~> 1.5", [hex: :ex2ms, repo: "hexpm", optional: false]}], "hexpm"},
@@ -38,7 +39,7 @@
   "hackney": {:hex, :hackney, "1.15.1", "9f8f471c844b8ce395f7b6d8398139e26ddca9ebc171a8b91342ee15a19963f4", [:rebar3], [{:certifi, "2.5.1", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "6.0.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.4", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"},
   "html_entities": {:hex, :html_entities, "0.4.0", "f2fee876858cf6aaa9db608820a3209e45a087c5177332799592142b50e89a6b", [:mix], [], "hexpm"},
   "html_sanitize_ex": {:hex, :html_sanitize_ex, "1.3.0", "f005ad692b717691203f940c686208aa3d8ffd9dd4bb3699240096a51fa9564e", [:mix], [{:mochiweb, "~> 2.15", [hex: :mochiweb, repo: "hexpm", optional: false]}], "hexpm"},
-  "http_signatures": {:git, "https://git.pleroma.social/pleroma/http_signatures.git", "a2a5982fa167fb1352fbd518ce6b606ba233a989", [ref: "a2a5982fa167fb1352fbd518ce6b606ba233a989"]},
+  "http_signatures": {:git, "https://git.pleroma.social/pleroma/http_signatures.git", "293d77bb6f4a67ac8bde1428735c3b42f22cbb30", [ref: "293d77bb6f4a67ac8bde1428735c3b42f22cbb30"]},
   "httpoison": {:hex, :httpoison, "1.2.0", "2702ed3da5fd7a8130fc34b11965c8cfa21ade2f232c00b42d96d4967c39a3a3", [:mix], [{:hackney, "~> 1.8", [hex: :hackney, repo: "hexpm", optional: false]}], "hexpm"},
   "idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"},
   "jason": {:hex, :jason, "1.1.2", "b03dedea67a99223a2eaf9f1264ce37154564de899fd3d8b9a21b1a6fd64afe7", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"},
@@ -56,22 +57,22 @@
   "nimble_parsec": {:hex, :nimble_parsec, "0.5.0", "90e2eca3d0266e5c53f8fbe0079694740b9c91b6747f2b7e3c5d21966bba8300", [:mix], [], "hexpm"},
   "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm"},
   "pbkdf2_elixir": {:hex, :pbkdf2_elixir, "0.12.3", "6706a148809a29c306062862c803406e88f048277f6e85b68faf73291e820b84", [:mix], [], "hexpm"},
-  "phoenix": {:hex, :phoenix, "1.4.8", "c72dc3adeb49c70eb963a0ea24f7a064ec1588e651e84e1b7ad5ed8253c0b4a2", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
+  "phoenix": {:hex, :phoenix, "1.4.9", "746d098e10741c334d88143d3c94cab1756435f94387a63441792e66ec0ee974", [:mix], [{:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:phoenix_pubsub, "~> 1.1", [hex: :phoenix_pubsub, repo: "hexpm", optional: false]}, {:plug, "~> 1.8.1 or ~> 1.9", [hex: :plug, repo: "hexpm", optional: false]}, {:plug_cowboy, "~> 1.0 or ~> 2.0", [hex: :plug_cowboy, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_ecto": {:hex, :phoenix_ecto, "4.0.0", "c43117a136e7399ea04ecaac73f8f23ee0ffe3e07acfcb8062fe5f4c9f0f6531", [:mix], [{:ecto, "~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:phoenix_html, "~> 2.9", [hex: :phoenix_html, repo: "hexpm", optional: true]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_html": {:hex, :phoenix_html, "2.13.1", "fa8f034b5328e2dfa0e4131b5569379003f34bc1fafdaa84985b0b9d2f12e68b", [:mix], [{:plug, "~> 1.5", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "phoenix_pubsub": {:hex, :phoenix_pubsub, "1.1.2", "496c303bdf1b2e98a9d26e89af5bba3ab487ba3a3735f74bf1f4064d2a845a3e", [:mix], [], "hexpm"},
   "pleroma_job_queue": {:hex, :pleroma_job_queue, "0.2.0", "879e660aa1cebe8dc6f0aaaa6aa48b4875e89cd961d4a585fd128e0773b31a18", [:mix], [], "hexpm"},
   "plug": {:hex, :plug, "1.8.2", "0bcce1daa420f189a6491f3940cc77ea7fb1919761175c9c3b59800d897440fc", [:mix], [{:mime, "~> 1.0", [hex: :mime, repo: "hexpm", optional: false]}, {:plug_crypto, "~> 1.0", [hex: :plug_crypto, repo: "hexpm", optional: false]}, {:telemetry, "~> 0.4", [hex: :telemetry, repo: "hexpm", optional: true]}], "hexpm"},
-  "plug_cowboy": {:hex, :plug_cowboy, "2.0.2", "6055f16868cc4882b24b6e1d63d2bada94fb4978413377a3b32ac16c18dffba2", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
+  "plug_cowboy": {:hex, :plug_cowboy, "2.1.0", "b75768153c3a8a9e8039d4b25bb9b14efbc58e9c4a6e6a270abff1cd30cbe320", [:mix], [{:cowboy, "~> 2.5", [hex: :cowboy, repo: "hexpm", optional: false]}, {:plug, "~> 1.7", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "plug_crypto": {:hex, :plug_crypto, "1.0.0", "18e49317d3fa343f24620ed22795ec29d4a5e602d52d1513ccea0b07d8ea7d4d", [:mix], [], "hexpm"},
   "plug_static_index_html": {:hex, :plug_static_index_html, "1.0.0", "840123d4d3975585133485ea86af73cb2600afd7f2a976f9f5fd8b3808e636a0", [:mix], [{:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"},
   "poison": {:hex, :poison, "3.1.0", "d9eb636610e096f86f25d9a46f35a9facac35609a7591b3be3326e99a0484665", [:mix], [], "hexpm"},
   "poolboy": {:hex, :poolboy, "1.5.2", "392b007a1693a64540cead79830443abf5762f5d30cf50bc95cb2c1aaafa006b", [:rebar3], [], "hexpm"},
   "postgrex": {:hex, :postgrex, "0.14.3", "5754dee2fdf6e9e508cbf49ab138df964278700b764177e8f3871e658b345a1e", [:mix], [{:connection, "~> 1.0", [hex: :connection, repo: "hexpm", optional: false]}, {:db_connection, "~> 2.0", [hex: :db_connection, repo: "hexpm", optional: false]}, {:decimal, "~> 1.5", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}], "hexpm"},
-  "prometheus": {:hex, :prometheus, "4.2.2", "a830e77b79dc6d28183f4db050a7cac926a6c58f1872f9ef94a35cd989aceef8", [:mix, :rebar3], [], "hexpm"},
+  "prometheus": {:hex, :prometheus, "4.4.1", "1e96073b3ed7788053768fea779cbc896ddc3bdd9ba60687f2ad50b252ac87d6", [:mix, :rebar3], [], "hexpm"},
   "prometheus_ecto": {:hex, :prometheus_ecto, "1.4.1", "6c768ea9654de871e5b32fab2eac348467b3021604ebebbcbd8bcbe806a65ed5", [:mix], [{:ecto, "~> 2.0 or ~> 3.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
   "prometheus_ex": {:hex, :prometheus_ex, "3.0.5", "fa58cfd983487fc5ead331e9a3e0aa622c67232b3ec71710ced122c4c453a02f", [:mix], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"},
-  "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.2.1", "964a74dfbc055f781d3a75631e06ce3816a2913976d1df7830283aa3118a797a", [:mix], [{:phoenix, "~> 1.3", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
+  "prometheus_phoenix": {:hex, :prometheus_phoenix, "1.3.0", "c4b527e0b3a9ef1af26bdcfbfad3998f37795b9185d475ca610fe4388fdd3bb5", [:mix], [{:phoenix, "~> 1.4", [hex: :phoenix, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.3 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}], "hexpm"},
   "prometheus_plugs": {:hex, :prometheus_plugs, "1.1.5", "25933d48f8af3a5941dd7b621c889749894d8a1082a6ff7c67cc99dec26377c5", [:mix], [{:accept, "~> 0.1", [hex: :accept, repo: "hexpm", optional: false]}, {:plug, "~> 1.0", [hex: :plug, repo: "hexpm", optional: false]}, {:prometheus_ex, "~> 1.1 or ~> 2.0 or ~> 3.0", [hex: :prometheus_ex, repo: "hexpm", optional: false]}, {:prometheus_process_collector, "~> 1.1", [hex: :prometheus_process_collector, repo: "hexpm", optional: true]}], "hexpm"},
   "prometheus_process_collector": {:hex, :prometheus_process_collector, "1.4.0", "6dbd39e3165b9ef1c94a7a820e9ffe08479f949dcdd431ed4aaea7b250eebfde", [:rebar3], [{:prometheus, "~> 4.0", [hex: :prometheus, repo: "hexpm", optional: false]}], "hexpm"},
   "quack": {:hex, :quack, "0.1.1", "cca7b4da1a233757fdb44b3334fce80c94785b3ad5a602053b7a002b5a8967bf", [:mix], [{:poison, ">= 1.0.0", [hex: :poison, repo: "hexpm", optional: false]}, {:tesla, "~> 1.2.0", [hex: :tesla, repo: "hexpm", optional: false]}], "hexpm"},

test/fixtures/rich_media/amz.html

@@ -0,0 +1,5 @@
+<meta name="twitter:card" content="summary" />
+<meta name="twitter:site" content="@flickr" />
+<meta name="twitter:title" content="Small Island Developing States Photo Submission" />
+<meta name="twitter:description" content="View the album on Flickr." />
+<meta name="twitter:image" content="https://pleroma.s3.ap-southeast-1.amazonaws.com/sachin%20%281%29%20_a%20-%25%2Aasdasd%20BNN%20bnnn%20.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIBLWWK6RGDQXDLJQ%2F20190716%2Fap-southeast-1%2Fs3%2Faws4_request&X-Amz-Date=20190716T175105Z&X-Amz-Expires=300000&X-Amz-Signature=04ffd6b98634f4b1bbabc62e0fac4879093cd54a6eed24fe8eb38e8369526bbf&X-Amz-SignedHeaders=host" />

test/fixtures/tesla_mock/kpherox@mstdn.jp.xml

@@ -0,0 +1,10 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<XRD xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
+  <Subject>acct:kPherox@mstdn.jp</Subject>
+  <Alias>https://mstdn.jp/@kPherox</Alias>
+  <Alias>https://mstdn.jp/users/kPherox</Alias>
+  <Link rel="http://webfinger.net/rel/profile-page" type="text/html" href="https://mstdn.jp/@kPherox"/>
+  <Link rel="http://schemas.google.com/g/2010#updates-from" type="application/atom+xml" href="https://mstdn.jp/users/kPherox.atom"/>
+  <Link rel="self" type="application/activity+json" href="https://mstdn.jp/users/kPherox"/>
+  <Link rel="http://ostatus.org/schema/1.0/subscribe" template="https://mstdn.jp/authorize_interaction?acct={uri}"/>
+</XRD>

test/fixtures/tesla_mock/wedistribute-article.json

@@ -0,0 +1,18 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams"
+  ],
+  "type": "Article",
+  "name": "The end is near: Mastodon plans to drop OStatus support",
+  "content": "<!-- wp:paragraph {\"dropCap\":true} -->\n<p class=\"has-drop-cap\">The days of OStatus are numbered. The venerable protocol has served as a glue between many different types of servers since the early days of the Fediverse, connecting StatusNet (now GNU Social) to Friendica, Hubzilla, Mastodon, and Pleroma.</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Now that many fediverse platforms support ActivityPub as a successor protocol, Mastodon appears to be drawing a line in the sand. In <a href=\"https://www.patreon.com/posts/mastodon-2-9-and-28121681\">a Patreon update</a>, Eugen Rochko writes:</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:quote -->\n<blockquote class=\"wp-block-quote\"><p>...OStatus...has overstayed its welcome in the code...and now that most of the network uses ActivityPub, it's time for it to go. </p><cite>Eugen Rochko, Mastodon creator</cite></blockquote>\n<!-- /wp:quote -->\n\n<!-- wp:paragraph -->\n<p>The <a href=\"https://github.com/tootsuite/mastodon/pull/11205\">pull request</a> to remove Pubsubhubbub and Salmon, two of the main components of OStatus, has already been merged into Mastodon's master branch.</p>\n<!-- /wp:paragraph -->\n\n<!-- wp:paragraph -->\n<p>Some projects will be left in the dark as a side effect of this. GNU Social and PostActiv, for example, both only communicate using OStatus. While <a href=\"https://mastodon.social/@dansup/102076573310057902\">some discussion</a> exists regarding adopting ActivityPub for GNU Social, and <a href=\"https://notabug.org/diogo/gnu-social/src/activitypub/plugins/ActivityPub\">a plugin is in development</a>, it hasn't been formally adopted yet. We just hope that the <a href=\"https://status.fsf.org/main/public\">Free Software Foundation's instance</a> gets updated in time!</p>\n<!-- /wp:paragraph -->",
+  "summary": "One of the largest platforms in the federated social web is dropping the protocol that it started with.",
+  "attributedTo": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+  "url": "https://wedistribute.org/2019/07/mastodon-drops-ostatus/",
+  "to": [
+    "https://www.w3.org/ns/activitystreams#Public",
+    "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers"
+  ],
+  "id": "https://wedistribute.org/wp-json/pterotype/v1/object/85810",
+  "likes": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/likes",
+  "shares": "https://wedistribute.org/wp-json/pterotype/v1/object/85810/shares"
+}

test/fixtures/tesla_mock/wedistribute-user.json

@@ -0,0 +1,31 @@
+{
+  "@context": [
+    "https://www.w3.org/ns/activitystreams",
+    "https://w3id.org/security/v1",
+    {
+      "manuallyApprovesFollowers": "as:manuallyApprovesFollowers"
+    }
+  ],
+  "type": "Organization",
+  "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+  "following": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/following",
+  "followers": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/followers",
+  "liked": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/liked",
+  "inbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/inbox",
+  "outbox": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog/outbox",
+  "name": "We Distribute",
+  "preferredUsername": "blog",
+  "summary": "<p>Connecting many threads in the federated web. We Distribute is an independent publication dedicated to the fediverse, decentralization, P2P technologies, and Free Software!</p>",
+  "url": "https://wedistribute.org/",
+  "publicKey": {
+    "id": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog#publicKey",
+    "owner": "https://wedistribute.org/wp-json/pterotype/v1/actor/-blog",
+    "publicKeyPem": "-----BEGIN PUBLIC KEY-----\r\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1bmUJ+y8PS8JFVi0KugN\r\nFl4pLvLog3V2lsV9ftmCXpveB/WJx66Tr1fQLsU3qYvQFc8UPGWD52zV4RENR1SN\r\nx0O6T2f97KUbRM+Ckow7Jyjtssgl+Mqq8UBZQ/+H8I/1Vpvt5E5hUykhFgwzx9qg\r\nzoIA3OK7alOpQbSoKXo0QcOh6yTRUnMSRMJAgUoZJzzXI/FmH/DtKr7ziQ1T2KWs\r\nVs8mWnTb/OlCxiheLuMlmJNMF+lPyVthvMIxF6Z5gV9d5QAmASSCI628e6uH2EUF\r\nDEEF5jo+Z5ffeNv28953lrnM+VB/wTjl3tYA+zCQeAmUPksX3E+YkXGxj+4rxBAY\r\n8wIDAQAB\r\n-----END PUBLIC KEY-----"
+  },
+  "manuallyApprovesFollowers": false,
+  "icon": {
+    "url": "https://wedistribute.org/wp-content/uploads/2019/02/b067de423757a538.png",
+    "type": "Image",
+    "mediaType": "image/png"
+  }
+}

test/flake_id_test.exs

@@ -39,4 +39,9 @@ defmodule Pleroma.FlakeIdTest do
     assert dump(flake_s) == {:ok, flake}
     assert dump(flake) == {:ok, flake}
   end
+
+  test "is_flake_id?/1" do
+    assert is_flake_id?("9eoozpwTul5mjSEDRI")
+    refute is_flake_id?("http://example.com/activities/3ebbadd1-eb14-4e20-8118-b6f79c0c7b0b")
+  end
 end

test/notification_test.exs

@@ -42,6 +42,28 @@ defmodule Pleroma.NotificationTest do
 
       assert notification.user_id == subscriber.id
     end
+
+    test "does not create a notification for subscribed users if status is a reply" do
+      user = insert(:user)
+      other_user = insert(:user)
+      subscriber = insert(:user)
+
+      User.subscribe(subscriber, other_user)
+
+      {:ok, activity} = CommonAPI.post(user, %{"status" => "test post"})
+
+      {:ok, _reply_activity} =
+        CommonAPI.post(other_user, %{
+          "status" => "test reply",
+          "in_reply_to_status_id" => activity.id
+        })
+
+      user_notifications = Notification.for_user(user)
+      assert length(user_notifications) == 1
+
+      subscriber_notifications = Notification.for_user(subscriber)
+      assert Enum.empty?(subscriber_notifications)
+    end
   end
 
   describe "create_notification" do
@@ -542,6 +564,64 @@ defmodule Pleroma.NotificationTest do
 
       assert Enum.empty?(Notification.for_user(user))
     end
+
+    test "notifications are deleted if a local user is deleted" do
+      user = insert(:user)
+      other_user = insert(:user)
+
+      {:ok, _activity} =
+        CommonAPI.post(user, %{"status" => "hi @#{other_user.nickname}", "visibility" => "direct"})
+
+      refute Enum.empty?(Notification.for_user(other_user))
+
+      User.delete(user)
+
+      assert Enum.empty?(Notification.for_user(other_user))
+    end
+
+    test "notifications are deleted if a remote user is deleted" do
+      remote_user = insert(:user)
+      local_user = insert(:user)
+
+      dm_message = %{
+        "@context" => "https://www.w3.org/ns/activitystreams",
+        "type" => "Create",
+        "actor" => remote_user.ap_id,
+        "id" => remote_user.ap_id <> "/activities/test",
+        "to" => [local_user.ap_id],
+        "cc" => [],
+        "object" => %{
+          "type" => "Note",
+          "content" => "Hello!",
+          "tag" => [
+            %{
+              "type" => "Mention",
+              "href" => local_user.ap_id,
+              "name" => "@#{local_user.nickname}"
+            }
+          ],
+          "to" => [local_user.ap_id],
+          "cc" => [],
+          "attributedTo" => remote_user.ap_id
+        }
+      }
+
+      {:ok, _dm_activity} = Transmogrifier.handle_incoming(dm_message)
+
+      refute Enum.empty?(Notification.for_user(local_user))
+
+      delete_user_message = %{
+        "@context" => "https://www.w3.org/ns/activitystreams",
+        "id" => remote_user.ap_id <> "/activities/delete",
+        "actor" => remote_user.ap_id,
+        "type" => "Delete",
+        "object" => remote_user.ap_id
+      }
+
+      {:ok, _delete_activity} = Transmogrifier.handle_incoming(delete_user_message)
+
+      assert Enum.empty?(Notification.for_user(local_user))
+    end
   end
 
   describe "for_user" do

test/object/fetcher_test.exs

@@ -110,6 +110,13 @@ defmodule Pleroma.Object.FetcherTest do
       assert object
     end
 
+    test "it can fetch wedistribute articles" do
+      {:ok, object} =
+        Fetcher.fetch_object_from_id("https://wedistribute.org/wp-json/pterotype/v1/object/85810")
+
+      assert object
+    end
+
     test "all objects with fake directions are rejected by the object fetcher" do
       assert {:error, _} =
                Fetcher.fetch_and_contain_remote_object_from_id(

test/plugs/authentication_plug_test.exs

@@ -8,6 +8,8 @@ defmodule Pleroma.Plugs.AuthenticationPlugTest do
   alias Pleroma.Plugs.AuthenticationPlug
   alias Pleroma.User
 
+  import ExUnit.CaptureLog
+
   setup %{conn: conn} do
     user = %User{
       id: 1,
@@ -64,19 +66,21 @@ defmodule Pleroma.Plugs.AuthenticationPlugTest do
       refute AuthenticationPlug.checkpw("test-password1", hash)
     end
 
+    @tag :skip_on_mac
     test "check sha512-crypt hash" do
       hash =
         "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
 
       assert AuthenticationPlug.checkpw("password", hash)
-      refute AuthenticationPlug.checkpw("password1", hash)
     end
 
     test "it returns false when hash invalid" do
       hash =
         "psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
 
-      refute Pleroma.Plugs.AuthenticationPlug.checkpw("password", hash)
+      assert capture_log(fn ->
+               refute Pleroma.Plugs.AuthenticationPlug.checkpw("password", hash)
+             end) =~ "[error] Password hash not recognized"
     end
   end
 end

test/plugs/http_signature_plug_test.exs

@@ -26,22 +26,4 @@ defmodule Pleroma.Web.Plugs.HTTPSignaturePlugTest do
       assert called(HTTPSignatures.validate_conn(:_))
     end
   end
-
-  test "bails out early if the signature isn't by the activity actor" do
-    params = %{"actor" => "https://mst3k.interlinked.me/users/luciferMysticus"}
-    conn = build_conn(:get, "/doesntmattter", params)
-
-    with_mock HTTPSignatures, validate_conn: fn _ -> false end do
-      conn =
-        conn
-        |> put_req_header(
-          "signature",
-          "keyId=\"http://mastodon.example.org/users/admin#main-key"
-        )
-        |> HTTPSignaturePlug.call(%{})
-
-      assert conn.assigns.valid_signature == false
-      refute called(HTTPSignatures.validate_conn(:_))
-    end
-  end
 end

test/plugs/legacy_authentication_plug_test.exs

@@ -5,19 +5,18 @@
 defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
   use Pleroma.Web.ConnCase
 
+  import Pleroma.Factory
+
   alias Pleroma.Plugs.LegacyAuthenticationPlug
   alias Pleroma.User
 
-  import Mock
-
   setup do
-    # password is "password"
-    user = %User{
-      id: 1,
-      name: "dude",
-      password_hash:
-        "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
-    }
+    user =
+      insert(:user,
+        password: "password",
+        password_hash:
+          "$6$9psBWV8gxkGOZWBz$PmfCycChoxeJ3GgGzwvhlgacb9mUoZ.KUXNCssekER4SJ7bOK53uXrHNb2e4i8yPFgSKyzaW9CcmrDXWIEMtD1"
+      )
 
     %{user: user}
   end
@@ -36,6 +35,7 @@ defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
     assert ret_conn == conn
   end
 
+  @tag :skip_on_mac
   test "it authenticates the auth_user if present and password is correct and resets the password",
        %{
          conn: conn,
@@ -46,22 +46,12 @@ defmodule Pleroma.Plugs.LegacyAuthenticationPlugTest do
       |> assign(:auth_credentials, %{username: "dude", password: "password"})
       |> assign(:auth_user, user)
 
-    conn =
-      with_mocks([
-        {:crypt, [], [crypt: fn _password, password_hash -> password_hash end]},
-        {User, [],
-         [
-           reset_password: fn user, %{password: password, password_confirmation: password} ->
-             {:ok, user}
-           end
-         ]}
-      ]) do
-        LegacyAuthenticationPlug.call(conn, %{})
-      end
+    conn = LegacyAuthenticationPlug.call(conn, %{})
 
-    assert conn.assigns.user == user
+    assert conn.assigns.user.id == user.id
   end
 
+  @tag :skip_on_mac
   test "it does nothing if the password is wrong", %{
     conn: conn,
     user: user

test/plugs/mapped_identity_to_signature_plug_test.exs

@@ -0,0 +1,59 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.Plugs.MappedSignatureToIdentityPlugTest do
+  use Pleroma.Web.ConnCase
+  alias Pleroma.Web.Plugs.MappedSignatureToIdentityPlug
+
+  import Tesla.Mock
+  import Plug.Conn
+
+  setup do
+    mock(fn env -> apply(HttpRequestMock, :request, [env]) end)
+    :ok
+  end
+
+  defp set_signature(conn, key_id) do
+    conn
+    |> put_req_header("signature", "keyId=\"#{key_id}\"")
+    |> assign(:valid_signature, true)
+  end
+
+  test "it successfully maps a valid identity with a valid signature" do
+    conn =
+      build_conn(:get, "/doesntmattter")
+      |> set_signature("http://mastodon.example.org/users/admin")
+      |> MappedSignatureToIdentityPlug.call(%{})
+
+    refute is_nil(conn.assigns.user)
+  end
+
+  test "it successfully maps a valid identity with a valid signature with payload" do
+    conn =
+      build_conn(:post, "/doesntmattter", %{"actor" => "http://mastodon.example.org/users/admin"})
+      |> set_signature("http://mastodon.example.org/users/admin")
+      |> MappedSignatureToIdentityPlug.call(%{})
+
+    refute is_nil(conn.assigns.user)
+  end
+
+  test "it considers a mapped identity to be invalid when it mismatches a payload" do
+    conn =
+      build_conn(:post, "/doesntmattter", %{"actor" => "http://mastodon.example.org/users/admin"})
+      |> set_signature("https://niu.moe/users/rye")
+      |> MappedSignatureToIdentityPlug.call(%{})
+
+    assert %{valid_signature: false} == conn.assigns
+  end
+
+  @tag skip: "known breakage; the testsuite presently depends on it"
+  test "it considers a mapped identity to be invalid when the identity cannot be found" do
+    conn =
+      build_conn(:post, "/doesntmattter", %{"actor" => "http://mastodon.example.org/users/admin"})
+      |> set_signature("http://niu.moe/users/rye")
+      |> MappedSignatureToIdentityPlug.call(%{})
+
+    assert %{valid_signature: false} == conn.assigns
+  end
+end

test/plugs/set_format_plug_test.exs

@@ -0,0 +1,38 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2017-2018 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Plugs.SetFormatPlugTest do
+  use ExUnit.Case, async: true
+  use Plug.Test
+
+  alias Pleroma.Plugs.SetFormatPlug
+
+  test "set format from params" do
+    conn =
+      :get
+      |> conn("/cofe?_format=json")
+      |> SetFormatPlug.call([])
+
+    assert %{format: "json"} == conn.assigns
+  end
+
+  test "set format from header" do
+    conn =
+      :get
+      |> conn("/cofe")
+      |> put_private(:phoenix_format, "xml")
+      |> SetFormatPlug.call([])
+
+    assert %{format: "xml"} == conn.assigns
+  end
+
+  test "doesn't set format" do
+    conn =
+      :get
+      |> conn("/cofe")
+      |> SetFormatPlug.call([])
+
+    refute conn.assigns[:format]
+  end
+end

test/signature_test.exs

@@ -5,6 +5,7 @@
 defmodule Pleroma.SignatureTest do
   use Pleroma.DataCase
 
+  import ExUnit.CaptureLog
   import Pleroma.Factory
   import Tesla.Mock
 
@@ -46,15 +47,15 @@ defmodule Pleroma.SignatureTest do
     end
 
     test "it returns error when not found user" do
-      assert Signature.fetch_public_key(make_fake_conn("test-ap_id")) ==
-               {:error, :error}
+      assert capture_log(fn ->
+               assert Signature.fetch_public_key(make_fake_conn("test-ap_id")) == {:error, :error}
+             end) =~ "[error] Could not decode user"
     end
 
     test "it returns error if public key is empty" do
       user = insert(:user, %{info: %{source_data: %{"publicKey" => %{}}}})
 
-      assert Signature.fetch_public_key(make_fake_conn(user.ap_id)) ==
-               {:error, :error}
+      assert Signature.fetch_public_key(make_fake_conn(user.ap_id)) == {:error, :error}
     end
   end
 
@@ -62,13 +63,14 @@ defmodule Pleroma.SignatureTest do
     test "it returns key" do
       ap_id = "https://mastodon.social/users/lambadalambda"
 
-      assert Signature.refetch_public_key(make_fake_conn(ap_id)) ==
-               {:ok, @rsa_public_key}
+      assert Signature.refetch_public_key(make_fake_conn(ap_id)) == {:ok, @rsa_public_key}
     end
 
     test "it returns error when not found user" do
-      assert Signature.refetch_public_key(make_fake_conn("test-ap_id")) ==
-               {:error, {:error, :ok}}
+      assert capture_log(fn ->
+               assert Signature.refetch_public_key(make_fake_conn("test-ap_id")) ==
+                        {:error, {:error, :ok}}
+             end) =~ "[error] Could not decode user"
     end
   end
 
@@ -100,4 +102,16 @@ defmodule Pleroma.SignatureTest do
              ) == {:error, []}
     end
   end
+
+  describe "key_id_to_actor_id/1" do
+    test "it properly deduces the actor id for misskey" do
+      assert Signature.key_id_to_actor_id("https://example.com/users/1234/publickey") ==
+               "https://example.com/users/1234"
+    end
+
+    test "it properly deduces the actor id for mastodon and pleroma" do
+      assert Signature.key_id_to_actor_id("https://example.com/users/1234#main-key") ==
+               "https://example.com/users/1234"
+    end
+  end
 end

test/support/factory.ex

@@ -118,17 +118,21 @@ defmodule Pleroma.Factory do
   def note_activity_factory(attrs \\ %{}) do
     user = attrs[:user] || insert(:user)
     note = attrs[:note] || insert(:note, user: user)
-    attrs = Map.drop(attrs, [:user, :note])
 
-    data = %{
-      "id" => Pleroma.Web.ActivityPub.Utils.generate_activity_id(),
-      "type" => "Create",
-      "actor" => note.data["actor"],
-      "to" => note.data["to"],
-      "object" => note.data["id"],
-      "published" => DateTime.utc_now() |> DateTime.to_iso8601(),
-      "context" => note.data["context"]
-    }
+    data_attrs = attrs[:data_attrs] || %{}
+    attrs = Map.drop(attrs, [:user, :note, :data_attrs])
+
+    data =
+      %{
+        "id" => Pleroma.Web.ActivityPub.Utils.generate_activity_id(),
+        "type" => "Create",
+        "actor" => note.data["actor"],
+        "to" => note.data["to"],
+        "object" => note.data["id"],
+        "published" => DateTime.utc_now() |> DateTime.to_iso8601(),
+        "context" => note.data["context"]
+      }
+      |> Map.merge(data_attrs)
 
     %Pleroma.Activity{
       data: data,

test/support/http_request_mock.ex

@@ -51,6 +51,10 @@ defmodule HttpRequestMock do
      }}
   end
 
+  def get("https://mastodon.social/users/not_found", _, _, _) do
+    {:ok, %Tesla.Env{status: 404}}
+  end
+
   def get("https://mastodon.sdf.org/users/rinpatch", _, _, _) do
     {:ok,
      %Tesla.Env{
@@ -301,6 +305,22 @@ defmodule HttpRequestMock do
      }}
   end
 
+  def get("https://wedistribute.org/wp-json/pterotype/v1/object/85810", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/wedistribute-article.json")
+     }}
+  end
+
+  def get("https://wedistribute.org/wp-json/pterotype/v1/actor/-blog", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/wedistribute-user.json")
+     }}
+  end
+
   def get("http://mastodon.example.org/users/admin", _, _, Accept: "application/activity+json") do
     {:ok,
      %Tesla.Env{
@@ -614,6 +634,15 @@ defmodule HttpRequestMock do
      }}
   end
 
+  def get(
+        "https://social.heldscal.la/.well-known/webfinger?resource=invalid_content@social.heldscal.la",
+        _,
+        _,
+        Accept: "application/xrd+xml,application/jrd+json"
+      ) do
+    {:ok, %Tesla.Env{status: 200, body: ""}}
+  end
+
   def get("http://framatube.org/.well-known/host-meta", _, _, _) do
     {:ok,
      %Tesla.Env{
@@ -915,6 +944,14 @@ defmodule HttpRequestMock do
     {:ok, %Tesla.Env{status: 404, body: ""}}
   end
 
+  def get("https://mstdn.jp/.well-known/webfinger?resource=acct:kpherox@mstdn.jp", _, _, _) do
+    {:ok,
+     %Tesla.Env{
+       status: 200,
+       body: File.read!("test/fixtures/tesla_mock/kpherox@mstdn.jp.xml")
+     }}
+  end
+
   def get(url, query, body, headers) do
     {:error,
      "Not implemented the mock response for get #{inspect(url)}, #{query}, #{inspect(body)}, #{

test/tasks/user_test.exs

@@ -5,6 +5,9 @@
 defmodule Mix.Tasks.Pleroma.UserTest do
   alias Pleroma.Repo
   alias Pleroma.User
+  alias Pleroma.Web.OAuth.Authorization
+  alias Pleroma.Web.OAuth.Token
+
   use Pleroma.DataCase
 
   import Pleroma.Factory
@@ -327,6 +330,13 @@ defmodule Mix.Tasks.Pleroma.UserTest do
       assert_received {:mix_shell, :info, [message]}
       assert message =~ "Invite for token #{invite.token} was revoked."
     end
+
+    test "it prints an error message when invite is not exist" do
+      Mix.Tasks.Pleroma.User.run(["revoke_invite", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No invite found"
+    end
   end
 
   describe "running delete_activities" do
@@ -337,6 +347,13 @@ defmodule Mix.Tasks.Pleroma.UserTest do
       assert_received {:mix_shell, :info, [message]}
       assert message == "User #{nickname} statuses deleted."
     end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["delete_activities", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
   end
 
   describe "running toggle_confirmed" do
@@ -364,6 +381,13 @@ defmodule Mix.Tasks.Pleroma.UserTest do
       refute user.info.confirmation_pending
       refute user.info.confirmation_token
     end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["toggle_confirmed", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
   end
 
   describe "search" do
@@ -386,4 +410,64 @@ defmodule Mix.Tasks.Pleroma.UserTest do
                User.Search.search("moon fediverse", for_user: user) |> Enum.map(& &1.id)
     end
   end
+
+  describe "signing out" do
+    test "it deletes all user's tokens and authorizations" do
+      user = insert(:user)
+      insert(:oauth_token, user: user)
+      insert(:oauth_authorization, user: user)
+
+      assert Repo.get_by(Token, user_id: user.id)
+      assert Repo.get_by(Authorization, user_id: user.id)
+
+      :ok = Mix.Tasks.Pleroma.User.run(["sign_out", user.nickname])
+
+      refute Repo.get_by(Token, user_id: user.id)
+      refute Repo.get_by(Authorization, user_id: user.id)
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["sign_out", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "No local user"
+    end
+  end
+
+  describe "tagging" do
+    test "it add tags to a user" do
+      user = insert(:user)
+
+      :ok = Mix.Tasks.Pleroma.User.run(["tag", user.nickname, "pleroma"])
+
+      user = User.get_cached_by_nickname(user.nickname)
+      assert "pleroma" in user.tags
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["tag", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "Could not change user tags"
+    end
+  end
+
+  describe "untagging" do
+    test "it deletes tags from a user" do
+      user = insert(:user, tags: ["pleroma"])
+      assert "pleroma" in user.tags
+
+      :ok = Mix.Tasks.Pleroma.User.run(["untag", user.nickname, "pleroma"])
+
+      user = User.get_cached_by_nickname(user.nickname)
+      assert Enum.empty?(user.tags)
+    end
+
+    test "it prints an error message when user is not exist" do
+      Mix.Tasks.Pleroma.User.run(["untag", "foo"])
+
+      assert_received {:mix_shell, :error, [message]}
+      assert message =~ "Could not change user tags"
+    end
+  end
 end

test/test_helper.exs

@@ -3,7 +3,8 @@
 # SPDX-License-Identifier: AGPL-3.0-only
 
 ExUnit.configure(formatters: [JUnitFormatter, ExUnit.CLIFormatter])
-ExUnit.start()
+os_exclude = if :os.type() == {:unix, :darwin}, do: [skip_on_mac: true], else: []
+ExUnit.start(exclude: os_exclude)
 Ecto.Adapters.SQL.Sandbox.mode(Pleroma.Repo, :manual)
 Mox.defmock(Pleroma.ReverseProxy.ClientMock, for: Pleroma.ReverseProxy.Client)
 {:ok, _} = Application.ensure_all_started(:ex_machina)

test/upload/filter/dedupe_test.exs

@@ -25,7 +25,7 @@ defmodule Pleroma.Upload.Filter.DedupeTest do
 
     assert {
              :ok,
-             %Pleroma.Upload{id: @shasum, path: "#{@shasum}.jpg"}
+             %Pleroma.Upload{id: @shasum, path: @shasum <> ".jpg"}
            } = Dedupe.filter(upload)
   end
 end

test/upload_test.exs

@@ -5,6 +5,8 @@
 defmodule Pleroma.UploadTest do
   use Pleroma.DataCase
 
+  import ExUnit.CaptureLog
+
   alias Pleroma.Upload
   alias Pleroma.Uploaders.Uploader
 
@@ -77,8 +79,12 @@ defmodule Pleroma.UploadTest do
 
     test "it returns error" do
       File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
-      assert Upload.store(@upload_file) == {:error, "Errors"}
-      Task.await(Agent.get(TestUploaderError, fn task_pid -> task_pid end))
+
+      assert capture_log(fn ->
+               assert Upload.store(@upload_file) == {:error, "Errors"}
+               Task.await(Agent.get(TestUploaderError, fn task_pid -> task_pid end))
+             end) =~
+               "[error] Elixir.Pleroma.Upload store (using Pleroma.UploadTest.TestUploaderError) failed: \"Errors\""
     end
   end
 
@@ -89,7 +95,11 @@ defmodule Pleroma.UploadTest do
 
     test "it returns error" do
       File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
-      assert Upload.store(@upload_file) == {:error, "Uploader callback timeout"}
+
+      assert capture_log(fn ->
+               assert Upload.store(@upload_file) == {:error, "Uploader callback timeout"}
+             end) =~
+               "[error] Elixir.Pleroma.Upload store (using Pleroma.UploadTest.TestUploader) failed: \"Uploader callback timeout\""
     end
   end
 
@@ -112,24 +122,6 @@ defmodule Pleroma.UploadTest do
       assert String.starts_with?(url, Pleroma.Web.base_url() <> "/media/")
     end
 
-    test "returns a media url with configured base_url" do
-      base_url = "https://cache.pleroma.social"
-
-      File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
-
-      file = %Plug.Upload{
-        content_type: "image/jpg",
-        path: Path.absname("test/fixtures/image_tmp.jpg"),
-        filename: "image.jpg"
-      }
-
-      {:ok, data} = Upload.store(file, base_url: base_url)
-
-      assert %{"url" => [%{"href" => url}]} = data
-
-      assert String.starts_with?(url, base_url <> "/media/")
-    end
-
     test "copies the file to the configured folder with deduping" do
       File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
 
@@ -256,4 +248,32 @@ defmodule Pleroma.UploadTest do
                "%3A%3F%23%5B%5D%40%21%24%26%5C%27%28%29%2A%2B%2C%3B%3D.jpg"
     end
   end
+
+  describe "Setting a custom base_url for uploaded media" do
+    setup do
+      Pleroma.Config.put([Pleroma.Upload, :base_url], "https://cache.pleroma.social")
+
+      on_exit(fn ->
+        Pleroma.Config.put([Pleroma.Upload, :base_url], nil)
+      end)
+    end
+
+    test "returns a media url with configured base_url" do
+      base_url = Pleroma.Config.get([Pleroma.Upload, :base_url])
+
+      File.cp!("test/fixtures/image.jpg", "test/fixtures/image_tmp.jpg")
+
+      file = %Plug.Upload{
+        content_type: "image/jpg",
+        path: Path.absname("test/fixtures/image_tmp.jpg"),
+        filename: "image.jpg"
+      }
+
+      {:ok, data} = Upload.store(file, base_url: base_url)
+
+      assert %{"url" => [%{"href" => url}]} = data
+
+      refute String.starts_with?(url, base_url <> "/media/")
+    end
+  end
 end

test/user_test.exs

@@ -824,6 +824,48 @@ defmodule Pleroma.UserTest do
       assert User.blocks?(user, collateral_user)
     end
 
+    test "does not block domain with same end" do
+      user = insert(:user)
+
+      collateral_user =
+        insert(:user, %{ap_id: "https://another-awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "awful-and-rude-instance.com")
+
+      refute User.blocks?(user, collateral_user)
+    end
+
+    test "does not block domain with same end if wildcard added" do
+      user = insert(:user)
+
+      collateral_user =
+        insert(:user, %{ap_id: "https://another-awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "*.awful-and-rude-instance.com")
+
+      refute User.blocks?(user, collateral_user)
+    end
+
+    test "blocks domain with wildcard for subdomain" do
+      user = insert(:user)
+
+      user_from_subdomain =
+        insert(:user, %{ap_id: "https://subdomain.awful-and-rude-instance.com/user/bully"})
+
+      user_with_two_subdomains =
+        insert(:user, %{
+          ap_id: "https://subdomain.second_subdomain.awful-and-rude-instance.com/user/bully"
+        })
+
+      user_domain = insert(:user, %{ap_id: "https://awful-and-rude-instance.com/user/bully"})
+
+      {:ok, user} = User.block_domain(user, "*.awful-and-rude-instance.com")
+
+      assert User.blocks?(user, user_from_subdomain)
+      assert User.blocks?(user, user_with_two_subdomains)
+      assert User.blocks?(user, user_domain)
+    end
+
     test "unblocks domains" do
       user = insert(:user)
       collateral_user = insert(:user, %{ap_id: "https://awful-and-rude-instance.com/user/bully"})
@@ -1327,4 +1369,28 @@ defmodule Pleroma.UserTest do
       assert User.is_internal_user?(user)
     end
   end
+
+  describe "update_and_set_cache/1" do
+    test "returns error when user is stale instead Ecto.StaleEntryError" do
+      user = insert(:user)
+
+      changeset = Ecto.Changeset.change(user, bio: "test")
+
+      Repo.delete(user)
+
+      assert {:error, %Ecto.Changeset{errors: [id: {"is stale", [stale: true]}], valid?: false}} =
+               User.update_and_set_cache(changeset)
+    end
+
+    test "performs update cache if user updated" do
+      user = insert(:user)
+      assert {:ok, nil} = Cachex.get(:user_cache, "ap_id:#{user.ap_id}")
+
+      changeset = Ecto.Changeset.change(user, bio: "test-bio")
+
+      assert {:ok, %User{bio: "test-bio"} = user} = User.update_and_set_cache(changeset)
+      assert {:ok, user} = Cachex.get(:user_cache, "ap_id:#{user.ap_id}")
+      assert %User{bio: "test-bio"} = User.get_cached_by_ap_id(user.ap_id)
+    end
+  end
 end

test/web/activity_pub/activity_pub_test.exs

@@ -6,11 +6,9 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do
   use Pleroma.DataCase
   alias Pleroma.Activity
   alias Pleroma.Builders.ActivityBuilder
-  alias Pleroma.Instances
   alias Pleroma.Object
   alias Pleroma.User
   alias Pleroma.Web.ActivityPub.ActivityPub
-  alias Pleroma.Web.ActivityPub.Publisher
   alias Pleroma.Web.ActivityPub.Utils
   alias Pleroma.Web.CommonAPI
 
@@ -1083,113 +1081,6 @@ defmodule Pleroma.Web.ActivityPub.ActivityPubTest do
            } = activity
   end
 
-  describe "publish_one/1" do
-    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is not specified",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is set",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: NaiveDateTime.utc_now()
-               })
-
-      assert called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_reachable` on successful federation if `unreachable_since` is nil",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: nil
-               })
-
-      refute called(Instances.set_reachable(inbox))
-    end
-
-    test_with_mock "calls `Instances.set_unreachable` on target inbox on non-2xx HTTP response code",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://404.site/users/nick1/inbox"
-
-      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "it calls `Instances.set_unreachable` on target inbox on request error of any kind",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://connrefused.site/users/nick1/inbox"
-
-      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      assert called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_unreachable` if target is reachable",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://200.site/users/nick1/inbox"
-
-      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
-
-      refute called(Instances.set_unreachable(inbox))
-    end
-
-    test_with_mock "does NOT call `Instances.set_unreachable` if target instance has non-nil `unreachable_since`",
-                   Instances,
-                   [:passthrough],
-                   [] do
-      actor = insert(:user)
-      inbox = "http://connrefused.site/users/nick1/inbox"
-
-      assert {:error, _} =
-               Publisher.publish_one(%{
-                 inbox: inbox,
-                 json: "{}",
-                 actor: actor,
-                 id: 1,
-                 unreachable_since: NaiveDateTime.utc_now()
-               })
-
-      refute called(Instances.set_unreachable(inbox))
-    end
-  end
-
   test "fetch_activities/2 returns activities addressed to a list " do
     user = insert(:user)
     member = insert(:user)

test/web/activity_pub/mrf/mrf_test.exs

@@ -0,0 +1,46 @@
+defmodule Pleroma.Web.ActivityPub.MRFTest do
+  use ExUnit.Case, async: true
+  alias Pleroma.Web.ActivityPub.MRF
+
+  test "subdomains_regex/1" do
+    assert MRF.subdomains_regex(["unsafe.tld", "*.unsafe.tld"]) == [
+             ~r/^unsafe.tld$/,
+             ~r/^(.*\.)*unsafe.tld$/
+           ]
+  end
+
+  describe "subdomain_match/2" do
+    test "common domains" do
+      regexes = MRF.subdomains_regex(["unsafe.tld", "unsafe2.tld"])
+
+      assert regexes == [~r/^unsafe.tld$/, ~r/^unsafe2.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "unsafe2.tld")
+
+      refute MRF.subdomain_match?(regexes, "example.com")
+    end
+
+    test "wildcard domains with one subdomain" do
+      regexes = MRF.subdomains_regex(["*.unsafe.tld"])
+
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "sub.unsafe.tld")
+      refute MRF.subdomain_match?(regexes, "anotherunsafe.tld")
+      refute MRF.subdomain_match?(regexes, "unsafe.tldanother")
+    end
+
+    test "wildcard domains with two subdomains" do
+      regexes = MRF.subdomains_regex(["*.unsafe.tld"])
+
+      assert regexes == [~r/^(.*\.)*unsafe.tld$/]
+
+      assert MRF.subdomain_match?(regexes, "unsafe.tld")
+      assert MRF.subdomain_match?(regexes, "sub.sub.unsafe.tld")
+      refute MRF.subdomain_match?(regexes, "sub.anotherunsafe.tld")
+      refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother")
+    end
+  end
+end

test/web/activity_pub/mrf/simple_policy_test.exs

@@ -49,6 +49,19 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :media_removal], ["*.remote.instance"])
+      media_message = build_media_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(media_message) ==
+               {:ok,
+                media_message
+                |> Map.put("object", Map.delete(media_message["object"], "attachment"))}
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   describe "when :media_nsfw" do
@@ -74,6 +87,20 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :media_nsfw], ["*.remote.instance"])
+      media_message = build_media_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(media_message) ==
+               {:ok,
+                media_message
+                |> put_in(["object", "tag"], ["foo", "nsfw"])
+                |> put_in(["object", "sensitive"], true)}
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   defp build_media_message do
@@ -106,6 +133,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
       assert SimplePolicy.filter(report_message) == {:reject, nil}
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :report_removal], ["*.remote.instance"])
+      report_message = build_report_message()
+      local_message = build_local_message()
+
+      assert SimplePolicy.filter(report_message) == {:reject, nil}
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
   end
 
   defp build_report_message do
@@ -146,6 +182,27 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
     end
 
+    test "match with wildcard domain" do
+      {actor, ftl_message} = build_ftl_actor_and_message()
+
+      ftl_message_actor_host =
+        ftl_message
+        |> Map.fetch!("actor")
+        |> URI.parse()
+        |> Map.fetch!(:host)
+
+      Config.put([:mrf_simple, :federated_timeline_removal], ["*." <> ftl_message_actor_host])
+      local_message = build_local_message()
+
+      assert {:ok, ftl_message} = SimplePolicy.filter(ftl_message)
+      assert actor.follower_address in ftl_message["to"]
+      refute actor.follower_address in ftl_message["cc"]
+      refute "https://www.w3.org/ns/activitystreams#Public" in ftl_message["to"]
+      assert "https://www.w3.org/ns/activitystreams#Public" in ftl_message["cc"]
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+    end
+
     test "has a matching host but only as:Public in to" do
       {_actor, ftl_message} = build_ftl_actor_and_message()
 
@@ -192,6 +249,14 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       assert SimplePolicy.filter(remote_message) == {:reject, nil}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :reject], ["*.remote.instance"])
+
+      remote_message = build_remote_message()
+
+      assert SimplePolicy.filter(remote_message) == {:reject, nil}
+    end
   end
 
   describe "when :accept" do
@@ -224,6 +289,16 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
       assert SimplePolicy.filter(local_message) == {:ok, local_message}
       assert SimplePolicy.filter(remote_message) == {:ok, remote_message}
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :accept], ["*.remote.instance"])
+
+      local_message = build_local_message()
+      remote_message = build_remote_message()
+
+      assert SimplePolicy.filter(local_message) == {:ok, local_message}
+      assert SimplePolicy.filter(remote_message) == {:ok, remote_message}
+    end
   end
 
   describe "when :avatar_removal" do
@@ -251,6 +326,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       refute filtered["icon"]
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :avatar_removal], ["*.remote.instance"])
+
+      remote_user = build_remote_user()
+      {:ok, filtered} = SimplePolicy.filter(remote_user)
+
+      refute filtered["icon"]
+    end
   end
 
   describe "when :banner_removal" do
@@ -278,6 +362,15 @@ defmodule Pleroma.Web.ActivityPub.MRF.SimplePolicyTest do
 
       refute filtered["image"]
     end
+
+    test "match with wildcard domain" do
+      Config.put([:mrf_simple, :banner_removal], ["*.remote.instance"])
+
+      remote_user = build_remote_user()
+      {:ok, filtered} = SimplePolicy.filter(remote_user)
+
+      refute filtered["image"]
+    end
   end
 
   defp build_local_message do

test/web/activity_pub/publisher_test.exs

@@ -0,0 +1,266 @@
+# Pleroma: A lightweight social networking server
+# Copyright © 2019 Pleroma Authors <https://pleroma.social/>
+# SPDX-License-Identifier: AGPL-3.0-only
+
+defmodule Pleroma.Web.ActivityPub.PublisherTest do
+  use Pleroma.DataCase
+
+  import Pleroma.Factory
+  import Tesla.Mock
+  import Mock
+
+  alias Pleroma.Activity
+  alias Pleroma.Instances
+  alias Pleroma.Web.ActivityPub.Publisher
+
+  @as_public "https://www.w3.org/ns/activitystreams#Public"
+
+  setup do
+    mock(fn env -> apply(HttpRequestMock, :request, [env]) end)
+    :ok
+  end
+
+  describe "determine_inbox/2" do
+    test "it returns sharedInbox for messages involving as:Public in to" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      activity = %Activity{
+        data: %{"to" => [@as_public], "cc" => [user.follower_address]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving as:Public in cc" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      activity = %Activity{
+        data: %{"cc" => [@as_public], "to" => [user.follower_address]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in to" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      user_two = insert(:user)
+      user_three = insert(:user)
+
+      activity = %Activity{
+        data: %{"cc" => [], "to" => [user.ap_id, user_two.ap_id, user_three.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in cc" do
+      user =
+        insert(:user, %{
+          info: %{source_data: %{"endpoints" => %{"sharedInbox" => "http://example.com/inbox"}}}
+        })
+
+      user_two = insert(:user)
+      user_three = insert(:user)
+
+      activity = %Activity{
+        data: %{"to" => [], "cc" => [user.ap_id, user_two.ap_id, user_three.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns sharedInbox for messages involving multiple recipients in total" do
+      user =
+        insert(:user, %{
+          info: %{
+            source_data: %{
+              "inbox" => "http://example.com/personal-inbox",
+              "endpoints" => %{"sharedInbox" => "http://example.com/inbox"}
+            }
+          }
+        })
+
+      user_two = insert(:user)
+
+      activity = %Activity{
+        data: %{"to" => [user_two.ap_id], "cc" => [user.ap_id]}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/inbox"
+    end
+
+    test "it returns inbox for messages involving single recipients in total" do
+      user =
+        insert(:user, %{
+          info: %{
+            source_data: %{
+              "inbox" => "http://example.com/personal-inbox",
+              "endpoints" => %{"sharedInbox" => "http://example.com/inbox"}
+            }
+          }
+        })
+
+      activity = %Activity{
+        data: %{"to" => [user.ap_id], "cc" => []}
+      }
+
+      assert Publisher.determine_inbox(activity, user) == "http://example.com/personal-inbox"
+    end
+  end
+
+  describe "publish_one/1" do
+    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is not specified",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "calls `Instances.set_reachable` on successful federation if `unreachable_since` is set",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: NaiveDateTime.utc_now()
+               })
+
+      assert called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_reachable` on successful federation if `unreachable_since` is nil",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: nil
+               })
+
+      refute called(Instances.set_reachable(inbox))
+    end
+
+    test_with_mock "calls `Instances.set_unreachable` on target inbox on non-2xx HTTP response code",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://404.site/users/nick1/inbox"
+
+      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "it calls `Instances.set_unreachable` on target inbox on request error of any kind",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://connrefused.site/users/nick1/inbox"
+
+      assert {:error, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      assert called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_unreachable` if target is reachable",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://200.site/users/nick1/inbox"
+
+      assert {:ok, _} = Publisher.publish_one(%{inbox: inbox, json: "{}", actor: actor, id: 1})
+
+      refute called(Instances.set_unreachable(inbox))
+    end
+
+    test_with_mock "does NOT call `Instances.set_unreachable` if target instance has non-nil `unreachable_since`",
+                   Instances,
+                   [:passthrough],
+                   [] do
+      actor = insert(:user)
+      inbox = "http://connrefused.site/users/nick1/inbox"
+
+      assert {:error, _} =
+               Publisher.publish_one(%{
+                 inbox: inbox,
+                 json: "{}",
+                 actor: actor,
+                 id: 1,
+                 unreachable_since: NaiveDateTime.utc_now()
+               })
+
+      refute called(Instances.set_unreachable(inbox))
+    end
+  end
+
+  describe "publish/2" do
+    test_with_mock "publishes an activity with BCC to all relevant peers.",
+                   Pleroma.Web.Federator.Publisher,
+                   [:passthrough],
+                   [] do
+      follower =
+        insert(:user,
+          local: false,
+          info: %{
+            ap_enabled: true,
+            source_data: %{"inbox" => "https://domain.com/users/nick1/inbox"}
+          }
+        )
+
+      actor = insert(:user, follower_address: follower.ap_id)
+      user = insert(:user)
+
+      {:ok, _follower_one} = Pleroma.User.follow(follower, actor)
+      actor = refresh_record(actor)
+
+      note_activity =
+        insert(:note_activity,
+          recipients: [follower.ap_id],
+          data_attrs: %{"bcc" => [user.ap_id]}
+        )
+
+      res = Publisher.publish(actor, note_activity)
+      assert res == :ok
+
+      assert called(
+               Pleroma.Web.Federator.Publisher.enqueue_one(Publisher, %{
+                 inbox: "https://domain.com/users/nick1/inbox",
+                 actor: actor,
+                 id: note_activity.data["id"]
+               })
+             )
+    end
+  end
+end

test/web/admin_api/admin_api_controller_test.exs

@@ -1010,6 +1010,17 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
                "uses" => 0
              }
     end
+
+    test "with invalid token" do
+      admin = insert(:user, info: %{is_admin: true})
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+        |> post("/api/pleroma/admin/users/revoke_invite", %{"token" => "foo"})
+
+      assert json_response(conn, :not_found) == "Not found"
+    end
   end
 
   describe "GET /api/pleroma/admin/reports/:id" do
@@ -1560,7 +1571,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
                 %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
                 %{"tuple" => [":seconds_valid", 60]},
                 %{"tuple" => [":path", ""]},
-                %{"tuple" => [":key1", nil]}
+                %{"tuple" => [":key1", nil]},
+                %{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}
               ]
             }
           ]
@@ -1576,7 +1588,8 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
                      %{"tuple" => [":method", "Pleroma.Captcha.Kocaptcha"]},
                      %{"tuple" => [":seconds_valid", 60]},
                      %{"tuple" => [":path", ""]},
-                     %{"tuple" => [":key1", nil]}
+                     %{"tuple" => [":key1", nil]},
+                     %{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}
                    ]
                  }
                ]
@@ -1902,6 +1915,103 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIControllerTest do
              }
     end
   end
+
+  describe "config mix tasks run" do
+    setup %{conn: conn} do
+      admin = insert(:user, info: %{is_admin: true})
+
+      temp_file = "config/test.exported_from_db.secret.exs"
+
+      Mix.shell(Mix.Shell.Quiet)
+
+      on_exit(fn ->
+        Mix.shell(Mix.Shell.IO)
+        :ok = File.rm(temp_file)
+      end)
+
+      dynamic = Pleroma.Config.get([:instance, :dynamic_configuration])
+
+      Pleroma.Config.put([:instance, :dynamic_configuration], true)
+
+      on_exit(fn ->
+        Pleroma.Config.put([:instance, :dynamic_configuration], dynamic)
+      end)
+
+      %{conn: assign(conn, :user, admin), admin: admin}
+    end
+
+    test "transfer settings to DB and to file", %{conn: conn, admin: admin} do
+      assert Pleroma.Repo.all(Pleroma.Web.AdminAPI.Config) == []
+      conn = get(conn, "/api/pleroma/admin/config/migrate_to_db")
+      assert json_response(conn, 200) == %{}
+      assert Pleroma.Repo.all(Pleroma.Web.AdminAPI.Config) > 0
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+        |> get("/api/pleroma/admin/config/migrate_from_db")
+
+      assert json_response(conn, 200) == %{}
+      assert Pleroma.Repo.all(Pleroma.Web.AdminAPI.Config) == []
+    end
+  end
+
+  describe "GET /api/pleroma/admin/users/:nickname/statuses" do
+    setup do
+      admin = insert(:user, info: %{is_admin: true})
+      user = insert(:user)
+
+      date1 = (DateTime.to_unix(DateTime.utc_now()) + 2000) |> DateTime.from_unix!()
+      date2 = (DateTime.to_unix(DateTime.utc_now()) + 1000) |> DateTime.from_unix!()
+      date3 = (DateTime.to_unix(DateTime.utc_now()) + 3000) |> DateTime.from_unix!()
+
+      insert(:note_activity, user: user, published: date1)
+      insert(:note_activity, user: user, published: date2)
+      insert(:note_activity, user: user, published: date3)
+
+      conn =
+        build_conn()
+        |> assign(:user, admin)
+
+      {:ok, conn: conn, user: user}
+    end
+
+    test "renders user's statuses", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses")
+
+      assert json_response(conn, 200) |> length() == 3
+    end
+
+    test "renders user's statuses with a limit", %{conn: conn, user: user} do
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses?page_size=2")
+
+      assert json_response(conn, 200) |> length() == 2
+    end
+
+    test "doesn't return private statuses by default", %{conn: conn, user: user} do
+      {:ok, _private_status} =
+        CommonAPI.post(user, %{"status" => "private", "visibility" => "private"})
+
+      {:ok, _public_status} =
+        CommonAPI.post(user, %{"status" => "public", "visibility" => "public"})
+
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses")
+
+      assert json_response(conn, 200) |> length() == 4
+    end
+
+    test "returns private statuses with godmode on", %{conn: conn, user: user} do
+      {:ok, _private_status} =
+        CommonAPI.post(user, %{"status" => "private", "visibility" => "private"})
+
+      {:ok, _public_status} =
+        CommonAPI.post(user, %{"status" => "public", "visibility" => "public"})
+
+      conn = get(conn, "/api/pleroma/admin/users/#{user.nickname}/statuses?godmode=true")
+
+      assert json_response(conn, 200) |> length() == 5
+    end
+  end
 end
 
 # Needed for testing

test/web/admin_api/config_test.exs

@@ -238,6 +238,14 @@ defmodule Pleroma.Web.AdminAPI.ConfigTest do
       assert Config.from_binary(binary) == [key: "value"]
     end
 
+    test "keyword with partial_chain key" do
+      binary =
+        Config.transform([%{"tuple" => [":partial_chain", "&:hackney_connect.partial_chain/1"]}])
+
+      assert binary == :erlang.term_to_binary(partial_chain: &:hackney_connect.partial_chain/1)
+      assert Config.from_binary(binary) == [partial_chain: &:hackney_connect.partial_chain/1]
+    end
+
     test "keyword" do
       binary =
         Config.transform([

test/web/common_api/common_api_utils_test.exs

@@ -360,4 +360,24 @@ defmodule Pleroma.Web.CommonAPI.UtilsTest do
       assert third_user.ap_id in to
     end
   end
+
+  describe "get_by_id_or_ap_id/1" do
+    test "get activity by id" do
+      activity = insert(:note_activity)
+      %Pleroma.Activity{} = note = Utils.get_by_id_or_ap_id(activity.id)
+      assert note.id == activity.id
+    end
+
+    test "get activity by ap_id" do
+      activity = insert(:note_activity)
+      %Pleroma.Activity{} = note = Utils.get_by_id_or_ap_id(activity.data["object"])
+      assert note.id == activity.id
+    end
+
+    test "get activity by object when type isn't `Create` " do
+      activity = insert(:like_activity)
+      %Pleroma.Activity{} = like = Utils.get_by_id_or_ap_id(activity.id)
+      assert like.data["object"] == activity.data["object"]
+    end
+  end
 end

test/web/fallback_test.exs

@@ -30,6 +30,10 @@ defmodule Pleroma.Web.FallbackTest do
            |> json_response(404) == %{"error" => "Not implemented"}
   end
 
+  test "GET /pleroma/admin -> /pleroma/admin/", %{conn: conn} do
+    assert redirected_to(get(conn, "/pleroma/admin")) =~ "/pleroma/admin/"
+  end
+
   test "GET /*path", %{conn: conn} do
     assert conn
            |> get("/foo")

test/web/federator_test.exs

@@ -22,6 +22,15 @@ defmodule Pleroma.Web.FederatorTest do
     :ok
   end
 
+  describe "Publisher.perform" do
+    test "call `perform` with unknown task" do
+      assert {
+               :error,
+               "Don't know what to do with this"
+             } = Pleroma.Web.Federator.Publisher.perform("test", :ok, :ok)
+    end
+  end
+
   describe "Publish an activity" do
     setup do
       user = insert(:user)

test/web/mastodon_api/account_view_test.exs

@@ -231,6 +231,16 @@ defmodule Pleroma.Web.MastodonAPI.AccountViewTest do
                AccountView.render("relationship.json", %{user: user, target: other_user})
     end
 
+    test "represent a relationship for the user blocking a domain" do
+      user = insert(:user)
+      other_user = insert(:user, ap_id: "https://bad.site/users/other_user")
+
+      {:ok, user} = User.block_domain(user, "bad.site")
+
+      assert %{domain_blocking: true, blocking: false} =
+               AccountView.render("relationship.json", %{user: user, target: other_user})
+    end
+
     test "represent a relationship for the user with a pending follow request" do
       user = insert(:user)
       other_user = insert(:user, %{info: %User.Info{locked: true}})

test/web/mastodon_api/mastodon_api_controller_test.exs

@@ -2815,11 +2815,11 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
       card_data = %{
         "image" => "http://ia.media-imdb.com/images/rock.jpg",
-        "provider_name" => "www.imdb.com",
-        "provider_url" => "http://www.imdb.com",
+        "provider_name" => "example.com",
+        "provider_url" => "https://example.com",
         "title" => "The Rock",
         "type" => "link",
-        "url" => "http://www.imdb.com/title/tt0117500/",
+        "url" => "https://example.com/ogp",
         "description" =>
           "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer.",
         "pleroma" => %{
@@ -2827,7 +2827,7 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
             "image" => "http://ia.media-imdb.com/images/rock.jpg",
             "title" => "The Rock",
             "type" => "video.movie",
-            "url" => "http://www.imdb.com/title/tt0117500/",
+            "url" => "https://example.com/ogp",
             "description" =>
               "Directed by Michael Bay. With Sean Connery, Nicolas Cage, Ed Harris, John Spencer."
           }
@@ -2868,14 +2868,14 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
                "title" => "Pleroma",
                "description" => "",
                "image" => nil,
-               "provider_name" => "pleroma.social",
-               "provider_url" => "https://pleroma.social",
-               "url" => "https://pleroma.social/",
+               "provider_name" => "example.com",
+               "provider_url" => "https://example.com",
+               "url" => "https://example.com/ogp-missing-data",
                "pleroma" => %{
                  "opengraph" => %{
                    "title" => "Pleroma",
                    "type" => "website",
-                   "url" => "https://pleroma.social/"
+                   "url" => "https://example.com/ogp-missing-data"
                  }
                }
              }
@@ -3154,6 +3154,21 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
       assert redirected_to(conn) == "/web/login"
     end
 
+    test "redirects not logged-in users to the login page on private instances", %{
+      conn: conn,
+      path: path
+    } do
+      is_public = Pleroma.Config.get([:instance, :public])
+      Pleroma.Config.put([:instance, :public], false)
+
+      conn = get(conn, path)
+
+      assert conn.status == 302
+      assert redirected_to(conn) == "/web/login"
+
+      Pleroma.Config.put([:instance, :public], is_public)
+    end
+
     test "does not redirect logged in users to the login page", %{conn: conn, path: path} do
       token = insert(:oauth_token)
 
@@ -3768,6 +3783,38 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
       assert Enum.empty?(response)
     end
+
+    test "does not return users who have favorited the status but are blocked", %{
+      conn: %{assigns: %{user: user}} = conn,
+      activity: activity
+    } do
+      other_user = insert(:user)
+      {:ok, user} = User.block(user, other_user)
+
+      {:ok, _, _} = CommonAPI.favorite(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/statuses/#{activity.id}/favourited_by")
+        |> json_response(:ok)
+
+      assert Enum.empty?(response)
+    end
+
+    test "does not fail on an unauthenticated request", %{conn: conn, activity: activity} do
+      other_user = insert(:user)
+      {:ok, _, _} = CommonAPI.favorite(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, nil)
+        |> get("/api/v1/statuses/#{activity.id}/favourited_by")
+        |> json_response(:ok)
+
+      [%{"id" => id}] = response
+      assert id == other_user.id
+    end
   end
 
   describe "GET /api/v1/statuses/:id/reblogged_by" do
@@ -3807,6 +3854,38 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
 
       assert Enum.empty?(response)
     end
+
+    test "does not return users who have reblogged the status but are blocked", %{
+      conn: %{assigns: %{user: user}} = conn,
+      activity: activity
+    } do
+      other_user = insert(:user)
+      {:ok, user} = User.block(user, other_user)
+
+      {:ok, _, _} = CommonAPI.repeat(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, user)
+        |> get("/api/v1/statuses/#{activity.id}/reblogged_by")
+        |> json_response(:ok)
+
+      assert Enum.empty?(response)
+    end
+
+    test "does not fail on an unauthenticated request", %{conn: conn, activity: activity} do
+      other_user = insert(:user)
+      {:ok, _, _} = CommonAPI.repeat(activity.id, other_user)
+
+      response =
+        conn
+        |> assign(:user, nil)
+        |> get("/api/v1/statuses/#{activity.id}/reblogged_by")
+        |> json_response(:ok)
+
+      [%{"id" => id}] = response
+      assert id == other_user.id
+    end
   end
 
   describe "POST /auth/password, with valid parameters" do
@@ -3859,4 +3938,45 @@ defmodule Pleroma.Web.MastodonAPI.MastodonAPIControllerTest do
       assert conn.resp_body == ""
     end
   end
+
+  describe "POST /api/v1/pleroma/accounts/confirmation_resend" do
+    setup do
+      setting = Pleroma.Config.get([:instance, :account_activation_required])
+
+      unless setting do
+        Pleroma.Config.put([:instance, :account_activation_required], true)
+        on_exit(fn -> Pleroma.Config.put([:instance, :account_activation_required], setting) end)
+      end
+
+      user = insert(:user)
+      info_change = User.Info.confirmation_changeset(user.info, need_confirmation: true)
+
+      {:ok, user} =
+        user
+        |> Changeset.change()
+        |> Changeset.put_embed(:info, info_change)
+        |> Repo.update()
+
+      assert user.info.confirmation_pending
+
+      [user: user]
+    end
+
+    test "resend account confirmation email", %{conn: conn, user: user} do
+      conn
+      |> assign(:user, user)
+      |> post("/api/v1/pleroma/accounts/confirmation_resend?email=#{user.email}")
+      |> json_response(:no_content)
+
+      email = Pleroma.Emails.UserEmail.account_confirmation_email(user)
+      notify_email = Pleroma.Config.get([:instance, :notify_email])
+      instance_name = Pleroma.Config.get([:instance, :name])
+
+      assert_email_sent(
+        from: {instance_name, notify_email},
+        to: {user.name, user.email},
+        html_body: email.html_body
+      )
+    end
+  end
 end

test/web/mastodon_api/search_controller_test.exs

@@ -24,12 +24,16 @@ defmodule Pleroma.Web.MastodonAPI.SearchControllerTest do
         {Pleroma.User, [], [search: fn _q, _o -> raise "Oops" end]},
         {Pleroma.Activity, [], [search: fn _u, _q, _o -> raise "Oops" end]}
       ] do
-        conn = get(conn, "/api/v2/search", %{"q" => "2hu"})
+        capture_log(fn ->
+          results =
+            conn
+            |> get("/api/v2/search", %{"q" => "2hu"})
+            |> json_response(200)
 
-        assert results = json_response(conn, 200)
-
-        assert results["accounts"] == []
-        assert results["statuses"] == []
+          assert results["accounts"] == []
+          assert results["statuses"] == []
+        end) =~
+          "[error] Elixir.Pleroma.Web.MastodonAPI.SearchController search error: %RuntimeError{message: \"Oops\"}"
       end
     end
 
@@ -99,14 +103,16 @@ defmodule Pleroma.Web.MastodonAPI.SearchControllerTest do
         {Pleroma.User, [], [search: fn _q, _o -> raise "Oops" end]},
         {Pleroma.Activity, [], [search: fn _u, _q, _o -> raise "Oops" end]}
       ] do
-        conn =
-          conn
-          |> get("/api/v1/search", %{"q" => "2hu"})
+        capture_log(fn ->
+          results =
+            conn
+            |> get("/api/v1/search", %{"q" => "2hu"})
+            |> json_response(200)
 
-        assert results = json_response(conn, 200)
-
-        assert results["accounts"] == []
-        assert results["statuses"] == []
+          assert results["accounts"] == []
+          assert results["statuses"] == []
+        end) =~
+          "[error] Elixir.Pleroma.Web.MastodonAPI.SearchController search error: %RuntimeError{message: \"Oops\"}"
       end
     end
 

test/web/mastodon_api/status_view_test.exs

@@ -300,6 +300,16 @@ defmodule Pleroma.Web.MastodonAPI.StatusViewTest do
     assert %{id: "2"} = StatusView.render("attachment.json", %{attachment: object})
   end
 
+  test "put the url advertised in the Activity in to the url attribute" do
+    id = "https://wedistribute.org/wp-json/pterotype/v1/object/85810"
+    [activity] = Activity.search(nil, id)
+
+    status = StatusView.render("status.json", %{activity: activity})
+
+    assert status.uri == id
+    assert status.url == "https://wedistribute.org/2019/07/mastodon-drops-ostatus/"
+  end
+
   test "a reblog" do
     user = insert(:user)
     activity = insert(:note_activity)

test/web/oauth/oauth_controller_test.exs

@@ -5,9 +5,7 @@
 defmodule Pleroma.Web.OAuth.OAuthControllerTest do
   use Pleroma.Web.ConnCase
   import Pleroma.Factory
-  import Mock
 
-  alias Pleroma.Registration
   alias Pleroma.Repo
   alias Pleroma.Web.OAuth.Authorization
   alias Pleroma.Web.OAuth.OAuthController
@@ -108,28 +106,26 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do
         "state" => ""
       }
 
-      with_mock Pleroma.Web.Auth.Authenticator,
-        get_registration: fn _ -> {:ok, registration} end do
-        conn =
-          get(
-            conn,
-            "/oauth/twitter/callback",
-            %{
-              "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
-              "oauth_verifier" => "QZl8vUqNvXMTKpdmUnGejJxuHG75WWWs",
-              "provider" => "twitter",
-              "state" => Poison.encode!(state_params)
-            }
-          )
+      conn =
+        conn
+        |> assign(:ueberauth_auth, %{provider: registration.provider, uid: registration.uid})
+        |> get(
+          "/oauth/twitter/callback",
+          %{
+            "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
+            "oauth_verifier" => "QZl8vUqNvXMTKpdmUnGejJxuHG75WWWs",
+            "provider" => "twitter",
+            "state" => Poison.encode!(state_params)
+          }
+        )
 
-        assert response = html_response(conn, 302)
-        assert redirected_to(conn) =~ ~r/#{redirect_uri}\?code=.+/
-      end
+      assert response = html_response(conn, 302)
+      assert redirected_to(conn) =~ ~r/#{redirect_uri}\?code=.+/
     end
 
     test "with user-unbound registration, GET /oauth/<provider>/callback renders registration_details page",
          %{app: app, conn: conn} do
-      registration = insert(:registration, user: nil)
+      user = insert(:user)
 
       state_params = %{
         "scope" => "read write",
@@ -138,26 +134,28 @@ defmodule Pleroma.Web.OAuth.OAuthControllerTest do
         "state" => "a_state"
       }
 
-      with_mock Pleroma.Web.Auth.Authenticator,
-        get_registration: fn _ -> {:ok, registration} end do
-        conn =
-          get(
-            conn,
-            "/oauth/twitter/callback",
-            %{
-              "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
-              "oauth_verifier" => "QZl8vUqNvXMTKpdmUnGejJxuHG75WWWs",
-              "provider" => "twitter",
-              "state" => Poison.encode!(state_params)
-            }
-          )
+      conn =
+        conn
+        |> assign(:ueberauth_auth, %{
+          provider: "twitter",
+          uid: "171799000",
+          info: %{nickname: user.nickname, email: user.email, name: user.name, description: nil}
+        })
+        |> get(
+          "/oauth/twitter/callback",
+          %{
+            "oauth_token" => "G-5a3AAAAAAAwMH9AAABaektfSM",
+            "oauth_verifier" => "QZl8vUqNvXMTKpdmUnGejJxuHG75WWWs",
+            "provider" => "twitter",
+            "state" => Poison.encode!(state_params)
+          }
+        )
 
-        assert response = html_response(conn, 200)
-        assert response =~ ~r/name="op" type="submit" value="register"/
-        assert response =~ ~r/name="op" type="submit" value="connect"/
-        assert response =~ Registration.email(registration)
-        assert response =~ Registration.nickname(registration)
-      end
+      assert response = html_response(conn, 200)
+      assert response =~ ~r/name="op" type="submit" value="register"/
+      assert response =~ ~r/name="op" type="submit" value="connect"/
+      assert response =~ user.email
+      assert response =~ user.nickname
     end
 
     test "on authentication error, GET /oauth/<provider>/callback redirects to `redirect_uri`", %{

test/web/ostatus/ostatus_controller_test.exs

@@ -4,7 +4,10 @@
 
 defmodule Pleroma.Web.OStatus.OStatusControllerTest do
   use Pleroma.Web.ConnCase
+
+  import ExUnit.CaptureLog
   import Pleroma.Factory
+
   alias Pleroma.Object
   alias Pleroma.User
   alias Pleroma.Web.CommonAPI
@@ -27,24 +30,28 @@ defmodule Pleroma.Web.OStatus.OStatusControllerTest do
       user = insert(:user)
       salmon = File.read!("test/fixtures/salmon.xml")
 
-      conn =
-        conn
-        |> put_req_header("content-type", "application/atom+xml")
-        |> post("/users/#{user.nickname}/salmon", salmon)
+      assert capture_log(fn ->
+               conn =
+                 conn
+                 |> put_req_header("content-type", "application/atom+xml")
+                 |> post("/users/#{user.nickname}/salmon", salmon)
 
-      assert response(conn, 200)
+               assert response(conn, 200)
+             end) =~ "[error]"
     end
 
     test "decodes a salmon with a changed magic key", %{conn: conn} do
       user = insert(:user)
       salmon = File.read!("test/fixtures/salmon.xml")
 
-      conn =
-        conn
-        |> put_req_header("content-type", "application/atom+xml")
-        |> post("/users/#{user.nickname}/salmon", salmon)
+      assert capture_log(fn ->
+               conn =
+                 conn
+                 |> put_req_header("content-type", "application/atom+xml")
+                 |> post("/users/#{user.nickname}/salmon", salmon)
 
-      assert response(conn, 200)
+               assert response(conn, 200)
+             end) =~ "[error]"
 
       # Set a wrong magic-key for a user so it has to refetch
       salmon_user = User.get_cached_by_ap_id("http://gs.example.org:4040/index.php/user/1")
@@ -61,12 +68,14 @@ defmodule Pleroma.Web.OStatus.OStatusControllerTest do
       |> Ecto.Changeset.put_embed(:info, info_cng)
       |> User.update_and_set_cache()
 
-      conn =
-        build_conn()
-        |> put_req_header("content-type", "application/atom+xml")
-        |> post("/users/#{user.nickname}/salmon", salmon)
+      assert capture_log(fn ->
+               conn =
+                 build_conn()
+                 |> put_req_header("content-type", "application/atom+xml")
+                 |> post("/users/#{user.nickname}/salmon", salmon)
 
-      assert response(conn, 200)
+               assert response(conn, 200)
+             end) =~ "[error]"
     end
   end
 
@@ -92,160 +101,538 @@ defmodule Pleroma.Web.OStatus.OStatusControllerTest do
     assert response(conn, 404)
   end
 
-  test "gets an object", %{conn: conn} do
-    note_activity = insert(:note_activity)
-    object = Object.normalize(note_activity)
-    user = User.get_cached_by_ap_id(note_activity.data["actor"])
-    [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
-    url = "/objects/#{uuid}"
+  describe "GET object/2" do
+    test "gets an object", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Object.normalize(note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
+      url = "/objects/#{uuid}"
+
+      conn =
+        conn
+        |> put_req_header("accept", "application/xml")
+        |> get(url)
+
+      expected =
+        ActivityRepresenter.to_simple_form(note_activity, user, true)
+        |> ActivityRepresenter.wrap_with_entry()
+        |> :xmerl.export_simple(:xmerl_xml)
+        |> to_string
+
+      assert response(conn, 200) == expected
+    end
+
+    test "redirects to /notice/id for html format", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Object.normalize(note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
+      url = "/objects/#{uuid}"
+
+      conn =
+        conn
+        |> put_req_header("accept", "text/html")
+        |> get(url)
+
+      assert redirected_to(conn) == "/notice/#{note_activity.id}"
+    end
+
+    test "500s when user not found", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Object.normalize(note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+      User.invalidate_cache(user)
+      Pleroma.Repo.delete(user)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
+      url = "/objects/#{uuid}"
+
+      conn =
+        conn
+        |> put_req_header("accept", "application/xml")
+        |> get(url)
+
+      assert response(conn, 500) == ~S({"error":"Something went wrong"})
+    end
+
+    test "404s on private objects", %{conn: conn} do
+      note_activity = insert(:direct_note_activity)
+      object = Object.normalize(note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
+
+      conn
+      |> get("/objects/#{uuid}")
+      |> response(404)
+    end
+
+    test "404s on nonexisting objects", %{conn: conn} do
+      conn
+      |> get("/objects/123")
+      |> response(404)
+    end
+  end
+
+  describe "GET activity/2" do
+    test "gets an activity in xml format", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
 
-    conn =
       conn
       |> put_req_header("accept", "application/xml")
-      |> get(url)
+      |> get("/activities/#{uuid}")
+      |> response(200)
+    end
 
-    expected =
-      ActivityRepresenter.to_simple_form(note_activity, user, true)
-      |> ActivityRepresenter.wrap_with_entry()
-      |> :xmerl.export_simple(:xmerl_xml)
-      |> to_string
+    test "redirects to /notice/id for html format", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
 
-    assert response(conn, 200) == expected
+      conn =
+        conn
+        |> put_req_header("accept", "text/html")
+        |> get("/activities/#{uuid}")
+
+      assert redirected_to(conn) == "/notice/#{note_activity.id}"
+    end
+
+    test "505s when user not found", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+      User.invalidate_cache(user)
+      Pleroma.Repo.delete(user)
+
+      conn =
+        conn
+        |> put_req_header("accept", "text/html")
+        |> get("/activities/#{uuid}")
+
+      assert response(conn, 500) == ~S({"error":"Something went wrong"})
+    end
+
+    test "404s on deleted objects", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Object.normalize(note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
+
+      conn
+      |> put_req_header("accept", "application/xml")
+      |> get("/objects/#{uuid}")
+      |> response(200)
+
+      Object.delete(object)
+
+      conn
+      |> put_req_header("accept", "application/xml")
+      |> get("/objects/#{uuid}")
+      |> response(404)
+    end
+
+    test "404s on private activities", %{conn: conn} do
+      note_activity = insert(:direct_note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
+
+      conn
+      |> get("/activities/#{uuid}")
+      |> response(404)
+    end
+
+    test "404s on nonexistent activities", %{conn: conn} do
+      conn
+      |> get("/activities/123")
+      |> response(404)
+    end
+
+    test "gets an activity in AS2 format", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
+      url = "/activities/#{uuid}"
+
+      conn =
+        conn
+        |> put_req_header("accept", "application/activity+json")
+        |> get(url)
+
+      assert json_response(conn, 200)
+    end
   end
 
-  test "404s on private objects", %{conn: conn} do
-    note_activity = insert(:direct_note_activity)
-    object = Object.normalize(note_activity)
-    [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
+  describe "GET notice/2" do
+    test "gets a notice in xml format", %{conn: conn} do
+      note_activity = insert(:note_activity)
 
-    conn
-    |> get("/objects/#{uuid}")
-    |> response(404)
-  end
+      conn
+      |> get("/notice/#{note_activity.id}")
+      |> response(200)
+    end
 
-  test "404s on nonexisting objects", %{conn: conn} do
-    conn
-    |> get("/objects/123")
-    |> response(404)
-  end
+    test "gets a notice in AS2 format", %{conn: conn} do
+      note_activity = insert(:note_activity)
 
-  test "gets an activity in xml format", %{conn: conn} do
-    note_activity = insert(:note_activity)
-    [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
-
-    conn
-    |> put_req_header("accept", "application/xml")
-    |> get("/activities/#{uuid}")
-    |> response(200)
-  end
-
-  test "404s on deleted objects", %{conn: conn} do
-    note_activity = insert(:note_activity)
-    object = Object.normalize(note_activity)
-    [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, object.data["id"]))
-
-    conn
-    |> put_req_header("accept", "application/xml")
-    |> get("/objects/#{uuid}")
-    |> response(200)
-
-    Object.delete(object)
-
-    conn
-    |> put_req_header("accept", "application/xml")
-    |> get("/objects/#{uuid}")
-    |> response(404)
-  end
-
-  test "404s on private activities", %{conn: conn} do
-    note_activity = insert(:direct_note_activity)
-    [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
-
-    conn
-    |> get("/activities/#{uuid}")
-    |> response(404)
-  end
-
-  test "404s on nonexistent activities", %{conn: conn} do
-    conn
-    |> get("/activities/123")
-    |> response(404)
-  end
-
-  test "gets a notice in xml format", %{conn: conn} do
-    note_activity = insert(:note_activity)
-
-    conn
-    |> get("/notice/#{note_activity.id}")
-    |> response(200)
-  end
-
-  test "gets a notice in AS2 format", %{conn: conn} do
-    note_activity = insert(:note_activity)
-
-    conn
-    |> put_req_header("accept", "application/activity+json")
-    |> get("/notice/#{note_activity.id}")
-    |> json_response(200)
-  end
-
-  test "only gets a notice in AS2 format for Create messages", %{conn: conn} do
-    note_activity = insert(:note_activity)
-    url = "/notice/#{note_activity.id}"
-
-    conn =
       conn
       |> put_req_header("accept", "application/activity+json")
-      |> get(url)
+      |> get("/notice/#{note_activity.id}")
+      |> json_response(200)
+    end
 
-    assert json_response(conn, 200)
+    test "500s when actor not found", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+      User.invalidate_cache(user)
+      Pleroma.Repo.delete(user)
 
-    user = insert(:user)
+      conn =
+        conn
+        |> get("/notice/#{note_activity.id}")
 
-    {:ok, like_activity, _} = CommonAPI.favorite(note_activity.id, user)
-    url = "/notice/#{like_activity.id}"
+      assert response(conn, 500) == ~S({"error":"Something went wrong"})
+    end
 
-    assert like_activity.data["type"] == "Like"
+    test "only gets a notice in AS2 format for Create messages", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      url = "/notice/#{note_activity.id}"
 
-    conn =
-      build_conn()
-      |> put_req_header("accept", "application/activity+json")
-      |> get(url)
+      conn =
+        conn
+        |> put_req_header("accept", "application/activity+json")
+        |> get(url)
 
-    assert response(conn, 404)
+      assert json_response(conn, 200)
+
+      user = insert(:user)
+
+      {:ok, like_activity, _} = CommonAPI.favorite(note_activity.id, user)
+      url = "/notice/#{like_activity.id}"
+
+      assert like_activity.data["type"] == "Like"
+
+      conn =
+        build_conn()
+        |> put_req_header("accept", "application/activity+json")
+        |> get(url)
+
+      assert response(conn, 404)
+    end
+
+    test "render html for redirect for html format", %{conn: conn} do
+      note_activity = insert(:note_activity)
+
+      resp =
+        conn
+        |> put_req_header("accept", "text/html")
+        |> get("/notice/#{note_activity.id}")
+        |> response(200)
+
+      assert resp =~
+               "<meta content=\"#{Pleroma.Web.base_url()}/notice/#{note_activity.id}\" property=\"og:url\">"
+
+      user = insert(:user)
+
+      {:ok, like_activity, _} = CommonAPI.favorite(note_activity.id, user)
+
+      assert like_activity.data["type"] == "Like"
+
+      resp =
+        conn
+        |> put_req_header("accept", "text/html")
+        |> get("/notice/#{like_activity.id}")
+        |> response(200)
+
+      assert resp =~ "<!--server-generated-meta-->"
+    end
+
+    test "404s a private notice", %{conn: conn} do
+      note_activity = insert(:direct_note_activity)
+      url = "/notice/#{note_activity.id}"
+
+      conn =
+        conn
+        |> get(url)
+
+      assert response(conn, 404)
+    end
+
+    test "404s a nonexisting notice", %{conn: conn} do
+      url = "/notice/123"
+
+      conn =
+        conn
+        |> get(url)
+
+      assert response(conn, 404)
+    end
   end
 
-  test "gets an activity in AS2 format", %{conn: conn} do
-    note_activity = insert(:note_activity)
-    [_, uuid] = hd(Regex.scan(~r/.+\/([\w-]+)$/, note_activity.data["id"]))
-    url = "/activities/#{uuid}"
+  describe "feed_redirect" do
+    test "undefined format. it redirects to feed", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
 
-    conn =
-      conn
-      |> put_req_header("accept", "application/activity+json")
-      |> get(url)
+      response =
+        conn
+        |> put_req_header("accept", "application/xml")
+        |> get("/users/#{user.nickname}")
+        |> response(302)
 
-    assert json_response(conn, 200)
+      assert response ==
+               "<html><body>You are being <a href=\"#{Pleroma.Web.base_url()}/users/#{
+                 user.nickname
+               }/feed.atom\">redirected</a>.</body></html>"
+    end
+
+    test "undefined format. it returns error when user not found", %{conn: conn} do
+      response =
+        conn
+        |> put_req_header("accept", "application/xml")
+        |> get("/users/jimm")
+        |> response(404)
+
+      assert response == ~S({"error":"Not found"})
+    end
+
+    test "activity+json format. it redirects on actual feed of user", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+
+      response =
+        conn
+        |> put_req_header("accept", "application/activity+json")
+        |> get("/users/#{user.nickname}")
+        |> json_response(200)
+
+      assert response["endpoints"] == %{
+               "oauthAuthorizationEndpoint" => "#{Pleroma.Web.base_url()}/oauth/authorize",
+               "oauthRegistrationEndpoint" => "#{Pleroma.Web.base_url()}/api/v1/apps",
+               "oauthTokenEndpoint" => "#{Pleroma.Web.base_url()}/oauth/token",
+               "sharedInbox" => "#{Pleroma.Web.base_url()}/inbox"
+             }
+
+      assert response["@context"] == [
+               "https://www.w3.org/ns/activitystreams",
+               "http://localhost:4001/schemas/litepub-0.1.jsonld",
+               %{"@language" => "und"}
+             ]
+
+      assert Map.take(response, [
+               "followers",
+               "following",
+               "id",
+               "inbox",
+               "manuallyApprovesFollowers",
+               "name",
+               "outbox",
+               "preferredUsername",
+               "summary",
+               "tag",
+               "type",
+               "url"
+             ]) == %{
+               "followers" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/followers",
+               "following" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/following",
+               "id" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}",
+               "inbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/inbox",
+               "manuallyApprovesFollowers" => false,
+               "name" => user.name,
+               "outbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/outbox",
+               "preferredUsername" => user.nickname,
+               "summary" => user.bio,
+               "tag" => [],
+               "type" => "Person",
+               "url" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}"
+             }
+    end
+
+    test "activity+json format. it returns error whe use not found", %{conn: conn} do
+      response =
+        conn
+        |> put_req_header("accept", "application/activity+json")
+        |> get("/users/jimm")
+        |> json_response(404)
+
+      assert response == "Not found"
+    end
+
+    test "json format. it redirects on actual feed of user", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+
+      response =
+        conn
+        |> put_req_header("accept", "application/json")
+        |> get("/users/#{user.nickname}")
+        |> json_response(200)
+
+      assert response["endpoints"] == %{
+               "oauthAuthorizationEndpoint" => "#{Pleroma.Web.base_url()}/oauth/authorize",
+               "oauthRegistrationEndpoint" => "#{Pleroma.Web.base_url()}/api/v1/apps",
+               "oauthTokenEndpoint" => "#{Pleroma.Web.base_url()}/oauth/token",
+               "sharedInbox" => "#{Pleroma.Web.base_url()}/inbox"
+             }
+
+      assert response["@context"] == [
+               "https://www.w3.org/ns/activitystreams",
+               "http://localhost:4001/schemas/litepub-0.1.jsonld",
+               %{"@language" => "und"}
+             ]
+
+      assert Map.take(response, [
+               "followers",
+               "following",
+               "id",
+               "inbox",
+               "manuallyApprovesFollowers",
+               "name",
+               "outbox",
+               "preferredUsername",
+               "summary",
+               "tag",
+               "type",
+               "url"
+             ]) == %{
+               "followers" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/followers",
+               "following" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/following",
+               "id" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}",
+               "inbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/inbox",
+               "manuallyApprovesFollowers" => false,
+               "name" => user.name,
+               "outbox" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}/outbox",
+               "preferredUsername" => user.nickname,
+               "summary" => user.bio,
+               "tag" => [],
+               "type" => "Person",
+               "url" => "#{Pleroma.Web.base_url()}/users/#{user.nickname}"
+             }
+    end
+
+    test "json format. it returns error whe use not found", %{conn: conn} do
+      response =
+        conn
+        |> put_req_header("accept", "application/json")
+        |> get("/users/jimm")
+        |> json_response(404)
+
+      assert response == "Not found"
+    end
+
+    test "html format. it redirects on actual feed of user", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      user = User.get_cached_by_ap_id(note_activity.data["actor"])
+
+      response =
+        conn
+        |> get("/users/#{user.nickname}")
+        |> response(200)
+
+      assert response ==
+               Fallback.RedirectController.redirector_with_meta(
+                 conn,
+                 %{user: user}
+               ).resp_body
+    end
+
+    test "html format. it returns error when user not found", %{conn: conn} do
+      response =
+        conn
+        |> get("/users/jimm")
+        |> json_response(404)
+
+      assert response == %{"error" => "Not found"}
+    end
   end
 
-  test "404s a private notice", %{conn: conn} do
-    note_activity = insert(:direct_note_activity)
-    url = "/notice/#{note_activity.id}"
+  describe "GET /notice/:id/embed_player" do
+    test "render embed player", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Pleroma.Object.normalize(note_activity)
 
-    conn =
-      conn
-      |> get(url)
+      object_data =
+        Map.put(object.data, "attachment", [
+          %{
+            "url" => [
+              %{
+                "href" =>
+                  "https://peertube.moe/static/webseed/df5f464b-be8d-46fb-ad81-2d4c2d1630e3-480.mp4",
+                "mediaType" => "video/mp4",
+                "type" => "Link"
+              }
+            ]
+          }
+        ])
 
-    assert response(conn, 404)
-  end
+      object
+      |> Ecto.Changeset.change(data: object_data)
+      |> Pleroma.Repo.update()
 
-  test "404s a nonexisting notice", %{conn: conn} do
-    url = "/notice/123"
+      conn =
+        conn
+        |> get("/notice/#{note_activity.id}/embed_player")
 
-    conn =
-      conn
-      |> get(url)
+      assert Plug.Conn.get_resp_header(conn, "x-frame-options") == ["ALLOW"]
 
-    assert response(conn, 404)
+      assert Plug.Conn.get_resp_header(
+               conn,
+               "content-security-policy"
+             ) == [
+               "default-src 'none';style-src 'self' 'unsafe-inline';img-src 'self' data: https:; media-src 'self' https:;"
+             ]
+
+      assert response(conn, 200) =~
+               "<video controls loop><source src=\"https://peertube.moe/static/webseed/df5f464b-be8d-46fb-ad81-2d4c2d1630e3-480.mp4\" type=\"video/mp4\">Your browser does not support video/mp4 playback.</video>"
+    end
+
+    test "404s when activity isn't create", %{conn: conn} do
+      note_activity = insert(:note_activity, data_attrs: %{"type" => "Like"})
+
+      assert conn
+             |> get("/notice/#{note_activity.id}/embed_player")
+             |> response(404)
+    end
+
+    test "404s when activity is direct message", %{conn: conn} do
+      note_activity = insert(:note_activity, data_attrs: %{"directMessage" => true})
+
+      assert conn
+             |> get("/notice/#{note_activity.id}/embed_player")
+             |> response(404)
+    end
+
+    test "404s when attachment is empty", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Pleroma.Object.normalize(note_activity)
+      object_data = Map.put(object.data, "attachment", [])
+
+      object
+      |> Ecto.Changeset.change(data: object_data)
+      |> Pleroma.Repo.update()
+
+      assert conn
+             |> get("/notice/#{note_activity.id}/embed_player")
+             |> response(404)
+    end
+
+    test "404s when attachment isn't audio or video", %{conn: conn} do
+      note_activity = insert(:note_activity)
+      object = Pleroma.Object.normalize(note_activity)
+
+      object_data =
+        Map.put(object.data, "attachment", [
+          %{
+            "url" => [
+              %{
+                "href" => "https://peertube.moe/static/webseed/480.jpg",
+                "mediaType" => "image/jpg",
+                "type" => "Link"
+              }
+            ]
+          }
+        ])
+
+      object
+      |> Ecto.Changeset.change(data: object_data)
+      |> Pleroma.Repo.update()
+
+      assert conn
+             |> get("/notice/#{note_activity.id}/embed_player")
+             |> response(404)
+    end
   end
 end