UBWA - subscribe_to_stream bug #374
Comments
|
Hi Pablo! I know the code part that is responsible for this and I want to rework it because of the Binance Websocket API. I want to send directly via loop.create_task() to reduce the latency, probably I can fix your issue as well. I will report here as soon as you can try it again. |
|
Hi Oliver, Please forgive my delayed response. It is the busiest time of the academic semester for me. The issue still exists as of today (version 2.4.0). From inspecting the ubwa manager class code statically, I think the issue of the False payload value may be localized here: If this is correct then I am curious as to why calls into this split_payload() method from the "unsubscribe" method do not result in the returned value of False (I have only observed the issue with subscribe attempts). Can you show me how calling into subscribe_to_stream() results in sending API requests via the established websocket connection? I cannot find the link between appending to self.stream_list[stream_id]['payload']—which is what I understand subscribe_to_stream() and unsubscribe_from_stream() to be doing (in the absence of this issue)—and the addition of a task to an event loop or some other mechanism to send an API request. In any case, if self.stream_list[stream_id]['payload'] does not get appended to from within subscribe_to_stream() whenever the issue is encountered, then doesn't that mean that no request gets sent to the Binance Websocket API? I can try to dynamically trace the issue when I have more time, if it would help, which would be after mid-May. |
|
Good approach from you, I will test why the I first have to integrate the option of accessing the event loop directly and sending it via websocket, which is not yet available. Currently the payload is stored on a stack and another process picks it up from the stack in the loop and then sends it (not optimal ^^) In the future I would like to have this done directly via a separate asyncio task. |
|
A legitimate reason for a Another reason would be the TypeError that is caught, here you should also find a log note! I have now equipped all the functions involved with logging at the crucial points, if you activate debug level "ERROR", you will have all the relevant information in the log before the |
|
Please test with version 2.6.0 and keep an eye on the max subscriptions! |
Hi Oliver, Thank you for looking into this. I keep an eye on the stream subscription count by printing the market count associated with the stream. Also, in the example script for this issue—ubwa_stream_subscribe_issue.py, we initialize markets to not exceed the subscription limit (I believe the value is 1024 markets), i.e., markets = markets[:subscribe_limit]. The subscribe_limit we grab from the method call indicated in your first provided web link. We then associate boolean values to these markets by sampling a uniform distribution and create the stream by subscribing to the markets associated with a value of True. We then iteratively unsubscribe from subscribed markets and subscribe to unsubscribed markets. So on average there will be 50% of the subscription number limit of markets subscribed to the stream, and never will there be more than the limit. Nevertheless, as an added measure, I will go ahead and use the get_number_of_subscriptions() method you reference with your second provided web-link. But my belief is that the problem isn't there. The TypeError, to my understanding, used to result when the program would attempt to iterate the payload variable, thereby assuming it to be of an iterable type. Yet, as we know, that payload variable is actually a Bool at times when the bug shows up. In version 2.4.0 the TypeError stopped getting thrown because of the try-catch wrap you placed before the program attempts to iterate the payload. As for a log note in either case, I will go ahead and run the example script until the bug appears, terminate the program, and inspect all logs in the generated log file. Thank you for extending the logger writes around more points surrounding this problem. I'll go ahead and activate the "ERROR" level of the logger when undergoing my testing. |
Ah, thank you for the explanation. I agree that an immediate addition of a task to a running event loop, as a result of a subscription or unsubscription method call, would be optimal. |
Yes, I will definitely do so. Please allow me to do so either Tuesday or Wednesday of the upcoming week. |
|
The limits are different "binance.com" supports 1024 while "binance.com-futures" only supports 200: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/wiki/Binance-websocket-endpoint-configuration-overview UBWA knows these limits and you can use |
|
It would actually be useful to throw an exception as soon as too many subscriptions are made. |
Agreed. Good idea |
I apologize, I should've specified. I was referring to "binance.com"; I've not tested for the issue in the futures markets. I'll make sure to keep calling get_number_of_free_subscription_slots() to verify that free slots remain available at times when the payload bug occurs. |
Please let me know if you still have problems or if I can close the issue. |
|
ubwa_stream_subscribe_issue_2.py.log Hi Oliver, I understand now why you desired to inspect the number of subscriptions to ensure we were not exceeding the maximum subscription number when encountering the error. Here is a mismatch for you regarding this topic: Print statements from the client side indicate that we do not exceed the subscription limit (1024 for Binance.com CEX): Yet, inspecting the log file we see that: 2024-05-16 01:00:23,629 [ERROR ] 3720 139648502327040 events: BinanceWebSocketApiManager.send_with_stream(a473242bad0d-11b3-4a3e-8141-92f4e838 - Timeout exceeded! So something is happening that is causing a "Too many subscriptions" error to be received by the websocket. UBWA has no awareness of this over-subscribing as indicated by the fact that, in subscribe_to_stream(), even though the logger reports that payload is None, it never reports that the limit of 1024 subscriptions per stream has been exceeded; in other words—if self.stream_list[stream_id]['subscriptions'] > self.max_subscriptions_per_stream:—is not getting hit. The payload value of None can be traced to the call to self.split_payload() here: which receives a None returned value coming from here: Since the logger reports a sequence of "BinanceWebSocketApiManager.send_with_stream(a473242bad0d-11b3-4a3e-8141-92f4e838 - Timeout exceeded!" errors before the "too many subscriptions" error, do you think that maybe self.stream_list[stream_id]['payload'] is getting overloaded from multiple calls into self.add_payload_to_stream() in the following ? |
|
I dont think its "overloaded", i will investigate this, but give me some time |
|
Yes of course, thank you for your time. |
Solution to Issue cannot be found in the documentation or other Issues and also occurs in the latest version of this library.
Version of this library.
2.4.0
Hardware?
VPS or other cloud hosting
Operating System?
Linux
Python version?
Python3.12
Installed wheel files
Installed packages
# Name Version Build Channel _libgcc_mutex 0.1 main _openmp_mutex 5.1 1_gnu aiohttp 3.9.3 pypi_0 pypi aiosignal 1.3.1 pypi_0 pypi aniso8601 9.0.1 pypi_0 pypi appdirs 1.4.4 pyhd3eb1b0_0 attrs 23.1.0 py312h06a4308_0 automat 20.2.0 py_0 bcrypt 3.2.0 py312h5eee18b_1 blas 1.0 mkl blinker 1.7.0 pypi_0 pypi bottleneck 1.3.7 py312ha883a20_0 brotli-python 1.0.9 py312h6a678d5_7 bzip2 1.0.8 h5eee18b_5 ca-certificates 2024.3.11 h06a4308_0 certifi 2024.2.2 py312h06a4308_0 cffi 1.16.0 py312h5eee18b_0 charset-normalizer 2.0.4 pyhd3eb1b0_0 cheroot 10.0.0 pypi_0 pypi click 8.1.7 pypi_0 pypi colorama 0.4.6 pypi_0 pypi constantly 23.10.4 py312h06a4308_0 cryptography 42.0.5 py312hdda0065_0 cython 3.0.9 pypi_0 pypi dateparser 1.2.0 pypi_0 pypi expat 2.5.0 h6a678d5_0 flask 3.0.2 pypi_0 pypi flask-restful 0.3.10 pypi_0 pypi frozenlist 1.4.1 pypi_0 pypi hyperlink 21.0.0 pyhd3eb1b0_0 idna 3.4 py312h06a4308_0 incremental 22.10.0 pyhd3eb1b0_0 intel-openmp 2023.1.0 hdb19cb5_46306 itsdangerous 2.1.2 pypi_0 pypi jaraco-functools 4.0.0 pypi_0 pypi jinja2 3.1.3 pypi_0 pypi ld_impl_linux-64 2.38 h1181459_1 libffi 3.4.4 h6a678d5_0 libgcc-ng 11.2.0 h1234567_1 libgomp 11.2.0 h1234567_1 libstdcxx-ng 11.2.0 h1234567_1 libuuid 1.41.5 h5eee18b_0 lucit-licensing-python 1.8.2 pypi_0 pypi markupsafe 2.1.5 pypi_0 pypi mkl 2023.1.0 h213fc3f_46344 mkl-service 2.4.0 py312h5eee18b_1 mkl_fft 1.3.8 py312h5eee18b_0 mkl_random 1.2.4 py312hdb19cb5_0 more-itertools 10.2.0 pypi_0 pypi multidict 6.0.5 pypi_0 pypi ncurses 6.4 h6a678d5_0 numexpr 2.8.7 py312hf827012_0 numpy 1.26.4 py312hc5e2394_0 numpy-base 1.26.4 py312h0da6c21_0 openssl 3.0.13 h7f8727e_0 pandas 2.1.1 py312h526ad5a_0 anaconda pip 23.3.1 py312h06a4308_0 psutil 5.9.8 pypi_0 pypi pyasn1 0.4.8 pyhd3eb1b0_0 pyasn1-modules 0.2.8 py_0 pycparser 2.21 pyhd3eb1b0_0 pycryptodome 3.20.0 pypi_0 pypi pyopenssl 24.0.0 py312h06a4308_0 pysocks 1.7.1 py312h06a4308_0 python 3.12.2 h996f2a0_0 python-binance 1.0.19 pypi_0 pypi python-dateutil 2.8.2 pyhd3eb1b0_0 python-tzdata 2023.3 pyhd3eb1b0_0 pytz 2023.3.post1 py312h06a4308_0 readline 8.2 h5eee18b_0 regex 2023.12.25 pypi_0 pypi requests 2.31.0 py312h06a4308_0 anaconda service_identity 18.1.0 pyhd3eb1b0_1 setuptools 68.2.2 py312h06a4308_0 shared-memory-dict 0.7.2 pypi_0 pypi simplejson 3.19.2 pypi_0 pypi six 1.16.0 pyhd3eb1b0_1 sqlite 3.41.2 h5eee18b_0 tbb 2021.8.0 hdb19cb5_0 tk 8.6.12 h1ccaba5_0 twisted 23.10.0 py312h06a4308_0 typing_extensions 4.9.0 py312h06a4308_1 tzdata 2024a h04d1e81_0 tzlocal 5.2 pypi_0 pypi ujson 5.9.0 pypi_0 pypi unicorn-binance-rest-api 2.2.1 pypi_0 pypi unicorn-binance-websocket-api 2.4.0 pypi_0 pypi unicorn-fy 1.14.1 pypi_0 pypi urllib3 1.26.18 py312h06a4308_0 websocket-client 1.7.0 pypi_0 pypi websockets 11.0.3 pypi_0 pypi werkzeug 3.0.1 pypi_0 pypi wheel 0.41.2 py312h06a4308_0 xz 5.4.6 h5eee18b_0 yarl 1.9.4 pypi_0 pypi zlib 1.2.13 h5eee18b_0 zope 1.0 py312h06a4308_1 zope.interface 5.4.0 py312h5eee18b_0Logging output
No response
Processing method?
process_stream_data
Used endpoint?
binance.com
Issue
ubwa_stream_subscribe_issue.py.log.zip
ubwa_stream_subscribe_issue.py.zip
periodic calls into subscribe_to_stream() returns a payload value of False, and the corresponding market does not undergo the subscription. This error happens periodically with no clear pattern as to when.
Print statements at the time of the first occurrence was as follows:
UNSUBSCRIBED btttrx, market count: 702
2024-04-13 23:30:16: FAILED TO SUBSCRIBE TO wintrx STREAM!!! Adding to self.unicorn_failed_market_subscriptions ...
Len of self.unicorn_failed_market_subscriptions: 1
UNSUBSCRIBED chzbnb, market count: 701
Log file simply shows that the coin in question, wintrx, was listed in the list of subscription markets, and subsequently, that the payload value was False.
The text was updated successfully, but these errors were encountered: