AFFiNE client supports connecting selfhosted server

[houjoe0829]

Desktop client will support connecting to selfhost servers.

This requires some work and We can't give you a specific time, but it may be included in our next major release.

[NeoHuncho]

Any news on the progress of this issue?

[affine-issue-bot]

Issue Status: 🏗 In Progress

🏗 In Progress

We’re currently working on the issue.

^{_{This is an automatic reply by the bot.}}

[mwhui]

is it possible to build my own cloud synchronization instead of using affine cloud for synchronization?

[LivioGama]

@mwhui if I'm not wrong, it's already the case, although I also find the wording confusing 😅 See here

[EYHN]

@LivioGama @mwhui Actually your selfhost affine has nothing to do with affine cloud.

All sync and account information are on the selfhost server.

I will adjust the wording of this document.

[the-chain-gamer]

this isn't completed yet, right? I cannot find a way to get my desktop client connecting to my docker instance.

[NeoHuncho]

Correct, it is not completed yet

[Felitendo]

I hope this gets added soon

[AndyPro720]

It's been around a year now, should be around the corner!

[richie80]

this would be great!

[easaw]

Looking forward too it!

I looked at AFFiNE a while back and seeing this has made me want to try it again :)

[HansAndreManfredson]

The web client is very capable, so there is no reason to don't give it a try!
I am using selfhosted AFFine and stopped using Logseq for it.

[almereyda]

Does this web client allow to collaborate with other people? And would you know how and where it persists its data?

Where do we find it to give it a try?

[SINCHOU]

Issue Status: 🏗 In Progress

🏗 In Progress

We’re currently working on the issue.

This is an automatic reply by the bot.

How about the progress
Really want to use the multi tab feature on desktop client

[ouchuji]

Any news on the progress of this issue?

[captorxxv]

any news ?

[forehalo]

follow up:

We're currently refactoring the code, one of which tasks is adding support for server settings. This is a prerequisite for enabling clients to connect to self-hosted instances.

[utopeadia]

Time flies, looking forward to the refactored client

[your-color]

I am looking forward to it!

[amrit92]

Eagerly waiting to try this out!

[Nikita-Ma]

• Eagerly waiting to try this out!

[LIHAQ]

I am looking forward to it!

[schromp]

Suggestion: Use this as a "Im waiting for this and am excited about it" button (by reacting).

I follow this post so i can see progress and the last few comments generated quite some noise.

[izmarel]

Eagerly waiting for this too. For me this feature has been the main blocker for starting to adopt Affine on a daily basis for drafting and documentation as I want to keep my data synced to my own server instance. Looking forward to it.

[larrydewey]

Any additional update on this? It looks like the original release was slotted to be included in 0.7, but Affine has recently release 0.18.1. Did this fall through the cracks?

[AndyPro720]

Honestly, if you look at affine selfhosted version, it lags a lot behind the release and in support. If you're planning to self-host primarily, in my opinion, better to look for alternatives.

[JokerQyou]

Honestly, if you look at affine selfhosted version, it lags a lot behind the release and in support. If you're planning to self-host primarily, in my opinion, better to look for alternatives.

For support it surely is, I think it's reasonable, since AFFiNE is still in active development, most developer manpower is probably invested to coding rather than documentation or support. However the docker image is not lagging too much. Take the current version which is referenced by the official docker-compose file:

which is exactly the version released yesterday:

[forehalo]

good news, 0.19.0 will make it available to connect your selfhost instances in desktop apps

[vishalkrv]

Thanks @forehalo

Found this commit in 0.19.0-canary.1 which enables the custom server approach for syc: 1fa1a95

[captorxxv]

Just tested with latest 0.19.0 and it works, i can connect to my own instance !!!

[nooz]

could someone define the steps to connect to local instance please? it is not immediately clear where and how to do this. thank you.

[AndyPro720]

Let the 0.19.0 release, simply pull it, and download the latest app. There's a straightforward option to connect them in the app. If you wish to do it now, you can use the canary release, but it may have quite a few bugs.

[Timtaran]

could someone define the steps to connect to local instance please? it is not immediately clear where and how to do this. thank you.

Simply enable Multiple Cloud Servers in the plugins tab in the settings
I'm not sure is it working well because I have stable version of affine on my server and I cannot see any workspaces in affine canary app

[ATPs]

any one succesful sync with self-hosted server? I can log in but not viewing the workspaces. failed in creating new workspaces from the client

[AtlasYang]

could someone define the steps to connect to local instance please? it is not immediately clear where and how to do this. thank you.

Simply enable Multiple Cloud Servers in the plugins tab in the settings I'm not sure is it working well because I have stable version of affine on my server and I cannot see any workspaces in affine canary app

I tried this method and got the same result. However, in the web app of my self-hosted instance, I can see the workspace created in the desktop app, but it doesn't appear in the desktop app itself.

[almereyda]

I'm not sure is it working well because I have stable version of affine on my server and I cannot see any workspaces in affine canary app

For reference, this is the configuration with Traefik sitting in an external web network that I came up with when trying to match the server version canary-cc21742with the commit of the canary app release v0.19.0-canary.3.

compose.yml

networks:
  internal:
  web:
    external: true

x-environment: &environment 
  - NODE_ENV=production
  - DATABASE_URL=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@postgres:5432/${POSTGRES_DB:-affine}
  - AFFINE_SERVER_EXTERNAL_URL
  - ENABLE_TELEMETRY=false

  - MAILER_HOST
  - MAILER_PORT
  - MAILER_USER
  - MAILER_PASSWORD
  - MAILER_SENDER
  - MAILER_SECURE

#  - OAUTH_OIDC_ISSUER
#  - OAUTH_OIDC_CLIENT_ID
#  - OAUTH_OIDC_CLIENT_SECRET
#  - OAUTH_OIDC_SCOPE
#  - OAUTH_OIDC_CLAIM_MAP_USERNAME
#  - OAUTH_OIDC_CLAIM_MAP_EMAIL
#  - OAUTH_OIDC_CLAIM_MAP_NAME

  - REDIS_SERVER_HOST
  - REDIS_SERVER_PORT
  - REDIS_SERVER_USER
  - REDIS_SERVER_PASSWORD
  - REDIS_SERVER_DATABASE


services:
  affine:
    image: ghcr.io/toeverything/affine-graphql:${AFFINE_REVISION:-stable}
    depends_on:
      redis:
        condition: service_healthy
      postgres:
        condition: service_healthy
      affine_migration:
        condition: service_completed_successfully
    volumes:
      # custom configurations
      - ${UPLOAD_LOCATION}:/root/.affine/storage
      - ${CONFIG_LOCATION}:/root/.affine/config
    environment: *environment
    restart: unless-stopped
    networks:
      - web
      - internal
    labels:
      traefik.enable: true
      traefik.http.services.io-allmende-affine.loadbalancer.server.port: "3010"
      traefik.http.routers.io-allmende-affine-web.entrypoints: web
      traefik.http.routers.io-allmende-affine-web.rule: Host(`${FQDN}`)
      traefik.http.routers.io-allmende-affine-web.middlewares: http-to-https
      traefik.http.middlewares.http-to-https.redirectscheme.scheme: https
      traefik.http.middlewares.http-to-https.redirectscheme.permanent: true
      traefik.http.routers.io-allmende-affine-webs.entrypoints: webs
      traefik.http.routers.io-allmende-affine-webs.rule: Host(`${FQDN}`)
      traefik.http.routers.io-allmende-affine-webs.tls: true
      traefik.http.routers.io-allmende-affine-webs.tls.certresolver: le


  affine_migration:
    image: ghcr.io/toeverything/affine-graphql:${AFFINE_REVISION:-stable}
    volumes:
      # custom configurations
      - ${UPLOAD_LOCATION}:/root/.affine/storage
      - ${CONFIG_LOCATION}:/root/.affine/config
    command: ['sh', '-c', 'node ./scripts/self-host-predeploy.js']
    environment: *environment
    depends_on:
      postgres:
        condition: service_healthy
    networks: ["web"]

  redis:
    image: redis:7-alpine
    command: '--save 60 1000 --appendonly yes --requirepass "${REDIS_SERVER_PASSWORD}"'
    restart: unless-stopped
    healthcheck:
      test: ['CMD-SHELL', 'redis-cli --pass "$${REDIS_SERVER_PASSWORD}" ping | grep PONG']
      start_period: 10s
      interval: 10s
      retries: 5
      timeout: 3s
    environment:
      - REDIS_SERVER_USER
      - REDIS_SERVER_PASSWORD
    volumes:
      - ${REDIS_DATA_LOCATION}:/data
    networks: ["web"]

  postgres:
    image: postgres:16-alpine
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    environment:
      - POSTGRES_DB
      - POSTGRES_USER
      - POSTGRES_PASSWORD
      - POSTGRES_INITDB_ARGS=--data-checksums
    healthcheck:
      test:
        ['CMD', 'pg_isready', '-U', '$${POSTGRES_USER}', '-d', '$${POSTGRES_DATABASE:-affine}']
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped
    networks: ["web"]

.env

FQDN=affine.example.org

# select a revision to deploy, available values: stable, beta, canary
#AFFINE_REVISION=stable
# AFFINE_REVISION=v0.19.0-canary.3 does not exist as OCI image tag
# https://github.com/toeverything/AFFiNE/releases/tag/v0.19.0-canary.3
# cc217425db0f54a1bebed67a33c764ee2dec5478
AFFINE_REVISION=canary-cc21742

# database credentials
POSTGRES_USER=org-example-affine
POSTGRES_PASSWORD=<openssl rand -base64 48>
POSTGRES_DB=org-example-affine

# position of the redis data to persist
REDIS_DATA_LOCATION=/data/bases/redis/affine.example.org
# position of the database data to persist
DB_DATA_LOCATION=/data/bases/postgres/affine.example.org
# position of the upload data(images, files, etc.) to persist
UPLOAD_LOCATION=/data/domains/affine.example.org/.state/storage
# position of the configuration files to persist
CONFIG_LOCATION=/data/domains/affine.example.org/.state/config
# set the port for the server container it will expose the server on
#PORT=3010

# set the host for the server for outgoing links
# AFFINE_SERVER_HTTPS=true
# AFFINE_SERVER_HOST=affine.yourdomain.com
# or 
AFFINE_SERVER_EXTERNAL_URL=https://affine.example.org

MAILER_HOST=mail.example.org
MAILER_PORT=465
[email protected]
MAILER_PASSWORD=
[email protected]
MAILER_SECURE=1

#OAUTH_OIDC_ISSUER=
#OAUTH_OIDC_CLIENT_ID=
#OAUTH_OIDC_CLIENT_SECRET=
#OAUTH_OIDC_SCOPE=
#OAUTH_OIDC_CLAIM_MAP_USERNAME=
#OAUTH_OIDC_CLAIM_MAP_EMAIL=
#OAUTH_OIDC_CLAIM_MAP_NAME=

REDIS_SERVER_HOST=redis
REDIS_SERVER_PORT=6379
REDIS_SERVER_USER=default
REDIS_SERVER_PASSWORD=<openssl rand -hex 32>
REDIS_SERVER_DATABASE=1

This yielded a working configuration, to which I could register the first administrative user at https://affine.example.org

It was then possible to connect an AFFiNE canary app with the self-hosted server.

The last canary release was retrieved from https://github.com/toeverything/AFFiNE/releases?q=canary&expanded=false with https://github.com/toeverything/AFFiNE/releases/tag/v0.19.0-canary.3 being the latest one. It references commit cc21742 who's short SHA is included in the container images, which are published at Packages · Toeverything.

They hold a series of canary releases, which can all be related to a specific git commit in the repository. Of which canary can always be newer than the last tagged canary pre-release.

• versions?filters%5Bversion_type%5D=tagged · affine-graphql versions · toeverything
  • 320954767?tag=canary-cc21742 · affine-graphql versions · toeverything
• versions?filters%5Bversion_type%5D=tagged · affine-front versions · toeverything
  • 320953545?tag=canary-cc21742 · affine-front versions · toeverything

Would someone be able to reproduce the setup?

One known issues is, that the workspace settings screen doesn't load, neither in App nor Web. There are two ever repeating error messages in the log:

[Nest] 1  - 12/17/2024, 9:39:08 PM   ERROR [GraphQL] Internal server error
GraphQLError: Unknown type "QueryChatHistoriesInput".
    at Object.NamedType (/app/node_modules/graphql/validation/rules/KnownTypeNamesRule.js:65:11)
    at Object.enter (/app/node_modules/graphql/language/visitor.js:301:32)
    at Object.enter (/app/node_modules/graphql/utilities/TypeInfo.js:391:27)
    at visit (/app/node_modules/graphql/language/visitor.js:197:21)
    at validate (/app/node_modules/graphql/validation/validate.js:91:24)
    at processGraphQLRequest (/app/node_modules/@apollo/server/dist/cjs/requestPipeline.js:101:61)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async internalExecuteOperation (/app/node_modules/@apollo/server/dist/cjs/ApolloServer.js:644:16)
    at async runHttpQuery (/app/node_modules/@apollo/server/dist/cjs/runHttpQuery.js:135:29)
    at async runPotentiallyBatchedHttpQuery (/app/node_modules/@apollo/server/dist/cjs/httpBatching.js:37:16)
[Nest] 1  - 12/17/2024, 9:39:08 PM   ERROR [GraphQL] Internal server error
GraphQLError: Cannot query field "copilot" on type "UserType".
    at Object.Field (/app/node_modules/graphql/validation/rules/FieldsOnCorrectTypeRule.js:51:13)
    at Object.enter (/app/node_modules/graphql/language/visitor.js:301:32)
    at Object.enter (/app/node_modules/graphql/utilities/TypeInfo.js:391:27)
    at visit (/app/node_modules/graphql/language/visitor.js:197:21)
    at validate (/app/node_modules/graphql/validation/validate.js:91:24)
    at processGraphQLRequest (/app/node_modules/@apollo/server/dist/cjs/requestPipeline.js:101:61)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async internalExecuteOperation (/app/node_modules/@apollo/server/dist/cjs/ApolloServer.js:644:16)
    at async runHttpQuery (/app/node_modules/@apollo/server/dist/cjs/runHttpQuery.js:135:29)
    at async runPotentiallyBatchedHttpQuery (/app/node_modules/@apollo/server/dist/cjs/httpBatching.js:37:16)

No AI API key is provided and AI support has been disabled in the app settings, for which a toggle already existed.

[Timtaran]

nowaying, 0.19 stable released, checking it rn

upd. there might be a problem with using 0.19 #9379

[forehalo]

Hello you all. As we promised, the 0.19.0 has been shipped on time, with the ability to connect to your own AFFiNE Server.

It's now introduced as an Experimental feature so we can't guarantee everything works as you expected. Feel free to report to us if you meet any bug.

Here is the instructions:

1. Get the latest desktop release at 0.19.0, this feature is desktop only
2. Open the [Experimental features] in setting tabs and you will see [Multiple Cloud Servers] toggle
3. Turn it on, and give it a refresh
4. You will see the Options on the Sign Up panel

[JokerQyou]

Checked with 0.19.1 (both server and app), Sign in to self-hosted instance does not work.

• The app sends loads of (over ~700) requests to sentry.io when I enter email and password. A total of over ~1000 requests after submitting the sign-in form, and at the bottom right of app it would pop up "You've signed in..." each time one of these requests finishes. Notice: this is after I disabled telemetry in About AFFiNE of the desktop app.
• After submitting the sign-in form, in devtools POST /sign-in shows 200 OK, and GET /api/auth/session and POST /graphql is also OK. But the sign-in form does not close. There is no further data loading.
• I can see my workspace name in About AFFiNE dialog, but clicking it shows empty loading animation.

Update:

If you delete the demo workspace in desktop app before signing in to self-hosted instance, the sign-in would stuck. If you have the same issue as mine, quit AFFiNE desktop app and start it again, during the app loading animation it would show a tab of demo workspace, and a plus icon on its right. Click the plus icon when you see it. This will add back the demo workspace. After the demo workspace is recovered, sign in would automatically finish and your cloud workspace will sync to the desktop app. After that you can safely delete the demo workspace from the app.