Possible to test Google Pub/Sub against real google project?

[titzel24]

Hello there 👋

I am evaluating Google Pub/Sub and I would need to implement consumers into my existing application using roadrunner server.

I found that you recently added native support for GPS ( https://docs.roadrunner.dev/docs/queues-and-jobs/google-pub-sub ) and wanted to try this with my real google cloud project.

But unfortunately I fail setting it up. I only get the following error in the console while booting the server:

{
  "level": "ERROR",
  "ts": "2025-03-18T05:08:12+0000",
  "logger": "jobs        ",
  "msg": "failed to initialize driver",
  "pipeline": "exchangerates",
  "driver": "google_pub_sub",
  "error": "google_pub_sub_consumer: context deadline exceeded"
}

There are no further error details visible.

In the docs you mention configuring a GPS host (example is set to 127.0.0.1:8085). I tried using pubsub.googleapis.com.

My config currently looks like this:

google_pub_sub:
  insecure: true
  host: pubsub.googleapis.com

jobs:
  pool:
    num_workers: 10

  pipelines:
    exchangeRates:
      driver: google_pub_sub
      config:
        priority: 1
        project_id: dev-66321
        topic: exchangeRate-modified
        max_delivery_attempts: 3

Is it even possible to use the current GPS beta integration with a real google project?

If yes, I would highly appreciate if somebody could help me setting it up correctly 🙂

[rustatian]

Hey @titzel24 👋🏻
I'm using a gcloud emulator to test the driver (you may check my docker-compose).
And config:

google_pub_sub:
  insecure: true
  host: 127.0.0.1:8085

Could you please try to use an emulator with that configuration?

[titzel24]

@rustatian Practically there is no error starting up the server when I use the emulator with the config you posted.

But I don't have any worker, too so I don't know whether it's possibly working or not.
When trying to add a worker I get the same error message like in my initial post.

jobs:
  pool:
    num_workers: 1

  pipelines:
    exchangeRates:
      driver: google_pub_sub
      config:
        priority: 1
        project_id: test
        topic: exchangeRate
        max_delivery_attempts: 3

I also created a topic and subscription named exchangeRate in the pubSub fake.

[rustatian]

I'm not sure I understand. Do you mean, you're trying to add a pool with workers to the JOBS plugin?

[titzel24]

I am trying to get a worker running in roadrunner consuming messages from google pub/sub.
I got this config from the official roadrunner documentation linked in my initial post.

[pheyse24]

When running roadrunner and the pubsub emulator in the same docker compose file,
(RoadRunner server started; version: 2024.3.5, buildtime: 2025-02-27T17:24:27+0000)

using the test scripts and dependencies from here: https://github.com/roadrunner-server/google-pub-sub/blob/master/tests/php_test_files/composer.json as command

using the same config above

google_pub_sub:
insecure: true
host: pubsub:8085

once we add a pipeline with that drivers, and we try to start roadrunner, it fails with the exact message
handle_serve_command: Function call error:
serve error from the plugin *jobs.Plugin stopping execution, error: jobs_plugin_serve: google_pub_sub_consumer: context deadline exceeded

As I don't know if logging can be more verbosly configured, I am out of ideas as on how to tweak the config.

Does the provided example in the driver code tests work for you on your local machine using docker compose for both, rr and pubsub emulator?
@rustatian

[rustatian]

I need to double-check that, but tests are executed on every commit, so from what I can say, at least in GitHub actions, everything works correctly.