[Bug]: inabox e2e testing does not work

[wawrzek]

What happened + What you expected to happen

I try to follow the README in the inabox folder, but the make run-e2e command fails. I expect it to succeed.

{"time":"2024-05-22T18:26:29.120717+01:00","level":"DEBUG","source":{"function":"github.com/Layr-Labs/eigenda/indexer.(*indexer).HandleAccumulator","file":"/Users/wawrzek/Ethereum/EigenLayer/eigenda/indexer/indexer.go","line":233},"msg":"Handling event","component":"Indexer","event":{"Type":"operator_socket_update","Payload":{"OperatorId":[251,57,10,100,18,45,179,149,127,178,32,195,196,45,95,113,233,122,176,201,149,218,78,30,92,195,38,22,2,218,197,39],"Socket":"localhost:32016;32017","Raw":{"address":"0xa82ff9afd8f496c3d6ac40e2a0f282e47488cfc9","topics":["0xec2963ab21c1e50e1e582aa542af2e4bf7bf38e6e1403c27b42e1c5d6e621eaa","0xfb390a64122db3957fb220c3c42d5f71e97ab0c995da4e1e5cc3261602dac527"],"data":"0x000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000156c6f63616c686f73743a33323031363b33323031370000000000000000000000","blockNumber":"0x5e","transactionHash":"0x9e445a8cb471364916e82681ee53ead76c04eb2110bd93e1468afc57d54f5d55","transactionIndex":"0x0","blockHash":"0x86a81ff82f5a182ed245e4fed8e52513e1c811093fb13ecd12160a92703a1d11","logIndex":"0x1","removed":false}}}}
{"time":"2024-05-22T18:26:29.120747+01:00","level":"DEBUG","source":{"function":"github.com/Layr-Labs/eigenda/indexer.(*indexer).HandleAccumulator","file":"/Users/wawrzek/Ethereum/EigenLayer/eigenda/indexer/indexer.go","line":233},"msg":"Handling event","component":"Indexer","event":{"Type":"operator_socket_update","Payload":{"OperatorId":[102,22,56,252,189,227,235,170,8,74,106,36,248,167,138,5,103,42,64,85,139,118,242,242,13,242,13,228,6,23,117,236],"Socket":"localhost:32006;32007","Raw":{"address":"0xa82ff9afd8f496c3d6ac40e2a0f282e47488cfc9","topics":["0xec2963ab21c1e50e1e582aa542af2e4bf7bf38e6e1403c27b42e1c5d6e621eaa","0x661638fcbde3ebaa084a6a24f8a78a05672a40558b76f2f20df20de4061775ec"],"data":"0x000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000156c6f63616c686f73743a33323030363b33323030370000000000000000000000","blockNumber":"0x5f","transactionHash":"0x002e0b87bba7366d531fdb147840c8735322980f0fe77fb4caa3b03f3d9e0d9a","transactionIndex":"0x0","blockHash":"0x293bc302c4e1faa4d0231e2240599c09aa184d18d4b3578466a5c0ad72c266e3","logIndex":"0x1","removed":false}}}}
{"time":"2024-05-22T18:26:29.120758+01:00","level":"DEBUG","source":{"function":"github.com/Layr-Labs/eigenda/indexer.(*indexer).HandleAccumulator","file":"/Users/wawrzek/Ethereum/EigenLayer/eigenda/indexer/indexer.go","line":233},"msg":"Handling event","component":"Indexer","event":{"Type":"operator_socket_update","Payload":{"OperatorId":[62,183,213,223,97,196,142,194,113,141,140,138,213,35,4,239,252,151,10,233,47,25,19,142,3,45,174,7,183,192,214,41],"Socket":"localhost:32011;32012","Raw":{"address":"0xa82ff9afd8f496c3d6ac40e2a0f282e47488cfc9","topics":["0xec2963ab21c1e50e1e582aa542af2e4bf7bf38e6e1403c27b42e1c5d6e621eaa","0x3eb7d5df61c48ec2718d8c8ad52304effc970ae92f19138e032dae07b7c0d629"],"data":"0x000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000156c6f63616c686f73743a33323031313b33323031320000000000000000000000","blockNumber":"0x5f","transactionHash":"0x68c69cbb716d41d826446e443602ded7e8d07c75561d00c28138380170fafcdc","transactionIndex":"0x1","blockHash":"0x293bc302c4e1faa4d0231e2240599c09aa184d18d4b3578466a5c0ad72c266e3","logIndex":"0xa","removed":false}}}}
------------------------------
• [FAILED] [10.001 seconds]
Inabox Integration [It] test end to end scenario
/Users/wawrzek/Ethereum/EigenLayer/eigenda/inabox/tests/integration_test.go:29

  [FAILED] Expected
      <*status.Error | 0x14000080480>:
      rpc error: code = DeadlineExceeded desc = context deadline exceeded
      {
          s: {
              s: {
                  state: {
                      NoUnkeyedLiterals: {},
                      DoNotCompare: [],
                      DoNotCopy: [],
                      atomicMessageInfo: nil,
                  },
                  sizeCache: 0,
                  unknownFields: nil,
                  Code: 4,
                  Message: "context deadline exceeded",
                  Details: nil,
              },
          },
      }
  to be nil
  In [It] at: /Users/wawrzek/Ethereum/EigenLayer/eigenda/inabox/tests/integration_test.go:54 @ 05/22/24 18:26:39.115
------------------------------
Stopping binaries
Stopping anvil
Stopping graph node
Stopping Dockertest resources
Expiring docker resource
Purging docker resource

Summarizing 1 Failure:
  [FAIL] Inabox Integration [It] test end to end scenario
  /Users/wawrzek/Ethereum/EigenLayer/eigenda/inabox/tests/integration_test.go:54

Ran 1 of 1 Specs in 409.497 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestInaboxIntegration (409.50s)
=== RUN   TestRatelimit
    ratelimit_test.go:172: Manual test for now
--- SKIP: TestRatelimit (0.00s)
FAIL
FAIL	github.com/Layr-Labs/eigenda/inabox/tests	409.768s
FAIL
make: *** [run-e2e] Error 1

Versions / Dependencies

EigenDA: master (commit cb32efd)
GoLang: go version go1.22.2 darwin/arm64
Os: MacOS (I see the same problem on Linux)

How to reproduce

• Download the source
• make build && cd inabox && make run-e2e

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[wawrzek]

I managed to make it work, by repeating all the steps from the GitHub Actions. Going to make a Docs PR.

[dungzvu]

I still fail to run the e2e test even when I follow all the steps from the GitHub Actions. Your PR does not help in this case.
My test result on commit 7806dfb on an ubuntu machine.

{"time":"2024-06-24T09:07:15.077490489Z","level":"DEBUG","source":{"function":"github.com/Layr-Labs/eigenda/indexer.(*indexer).HandleAccumulator","file":"/home/ubuntu/Layr-Labs/eigenda/indexer/indexer.go","line":233},"msg":"Handling event","component":"Indexer","event":{"Type":"operator_socket_update","Payload":{"OperatorId":[248,185,184,117,137,138,66,90,251,88,6,246,92,175,161,182,80,190,226,60,170,137,227,43,213,204,100,169,11,150,58,8],"Socket":"localhost:32021;32022","Raw":{"address":"0x02e8910b3b89690d4aec9fcc0ae2cd16fb6a4828","topics":["0xec2963ab21c1e50e1e582aa542af2e4bf7bf38e6e1403c27b42e1c5d6e621eaa","0xf8b9b875898a425afb5806f65cafa1b650bee23caa89e32bd5cc64a90b963a08"],"data":"0x000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000000156c6f63616c686f73743a33323032313b33323032320000000000000000000000","blockNumber":"0x36c","transactionHash":"0xe712c81584bbbd20419b6db258d9d8e2dbd4c607760b88a890a25ff2b9bee8b8","transactionIndex":"0x0","blockHash":"0xe4a88e1f9c3853f21fc4b88d55aef252b42fb36d137321e963904d82eda87ab3","logIndex":"0x1","removed":false}}}}
------------------------------
• [FAILED] [15.001 seconds]
Inabox Integration [It] test end to end scenario
/home/ubuntu/Layr-Labs/eigenda/inabox/tests/integration_test.go:29

  [FAILED] timed out
  In [It] at: /home/ubuntu/Layr-Labs/eigenda/inabox/tests/integration_test.go:74 @ 06/24/24 09:07:29.897
------------------------------
Stopping binaries
  rpc: http://localhost:8545
  verifyContracts: false
Stopping anvil
Stopping graph node
Stopping Dockertest resources
Expiring docker resource
Purging docker resource

Summarizing 1 Failure:
  [FAIL] Inabox Integration [It] test end to end scenario
  /home/ubuntu/Layr-Labs/eigenda/inabox/tests/integration_test.go:74

Ran 1 of 1 Specs in 102.495 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestInaboxIntegration (102.50s)
=== RUN   TestRatelimit
    ratelimit_test.go:172: Manual test for now
--- SKIP: TestRatelimit (0.00s)
FAIL
FAIL    github.com/Layr-Labs/eigenda/inabox/tests       102.517s
FAIL
make: *** [Makefile:49: run-e2e] Error 1

I got this error in batcher0.log

{"time":"2024-06-24T09:22:03.565400665Z","level":"ERROR","source":{"function":"github.com/Layr-Labs/eigenda/core/thegraph.(*indexedChainState).getQuorumAPK","file":"/home/ubuntu/Layr-Labs/eigenda/core/thegraph/state.go","line":252},"msg":"no quorum APK found for quorum 0, block number 1027","component":"IndexedChainState"}

I have no idea what I missed, or the doc is so poor.

[l0g2]

Hi @dungzvu ,

I also encountered the "no quorum APK found" error you mentioned, while using the release/0.8.3 branch. I managed to resolve it by cleaning up the environment and rerunning the tests. Here’s what I did:

1. Executed ./bin.sh stop to clean up the executable files.
2. Ran make stop-infra to clean up the infrastructure (including Anvil, Graph, and Docker).
3. Ran rm -r inabox/testdata/ to delete the test data directory.
4. Used ps aux | grep -E 'churner|disperser|retriever|bin/node' to list any remaining processes and terminated them one by one.

I hope this helps you as well!

update

I found that after reboot, I encounter the same error on the first run. It's necessary to follow the steps mentioned above to clean the environment and rerun the tests, in order to succeed.