-
Notifications
You must be signed in to change notification settings - Fork 1
/
tag-tools:linux:apline.html
100 lines (98 loc) · 4.91 KB
/
tag-tools:linux:apline.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="alternate"
type="application/rss+xml"
href="https://chenyo-17.github.io/org-static-blog/rss.xml"
title="RSS feed for https://chenyo-17.github.io/org-static-blog">
<title>org-static-blog</title>
</head>
<body>
<div id="preamble" class="status"></div>
<div id="content">
<h1 class="title">Posts tagged "tools:linux:apline":</h1>
<div class="post-date">24 Jun 2024</div><h1 class="post-title"><a href="https://chenyo-17.github.io/org-static-blog/2024-06-24-a-stupid-debugging-experience.html">A stupid debugging experience</a></h1>
<nav id="table-of-contents" role="doc-toc">
<h2>Table of Contents</h2>
<div id="text-table-of-contents" role="doc-toc">
<ul>
<li><a href="#orgf727f8f">1. What happended</a></li>
<li><a href="#orgfd53298">2. What did I do</a></li>
<li><a href="#org0423d38">3. Another issue of running RPC in docker</a></li>
</ul>
</div>
</nav>
<div id="outline-container-orgf727f8f" class="outline-2">
<h2 id="orgf727f8f"><span class="section-number-2">1.</span> What happended</h2>
<div class="outline-text-2" id="text-1">
<ul class="org-ul">
<li>Servers SA and SB have the same docker installation, and the same running container CA and CB.</li>
<li><p>
A Go file G can be built on CA, but on CB it reports this error:
</p>
<p class="verse">
runtime: failed to create new OS thread (have 2 already; errno=11)<br>
runtime: may need to increase max user processes (ulimit -u)<br>
fatal error: newosproc<br>
</p></li>
</ul>
</div>
</div>
<div id="outline-container-orgfd53298" class="outline-2">
<h2 id="orgfd53298"><span class="section-number-2">2.</span> What did I do</h2>
<div class="outline-text-2" id="text-2">
<ol class="org-ol">
<li>I compared any related configurations between SA and SB. and between CA and CB, e.g., <code class="src src-bash"><span style="color: #c678dd;">ulimit</span> -a</code>, <code class="src src-bash">/etc/security/limits.conf</code>. They all look the same.</li>
<li>I created a new container CN on SA with the same docker image, CN can compile G.</li>
<li>I looked into the (complex) <code>docker run</code> script for CA/CB and figured out it was due to a resource constraint <code>--pids-limit 100</code>.
<ul class="org-ul">
<li>Increasing this limit to 200 seems resolve the issue, but I had no idea why the Go compiler needed so many resources (perhaps due to package I imported).</li>
</ul></li>
<li><b><b>Until this point</b></b>, I realized, since the container did not support the compilation, why not just only transfer the compiled binary!
<ul class="org-ul">
<li>How silly that I didn’t even try this in the beginning!</li>
</ul></li>
<li>Since the program imports the <code>net</code> package, and there is a <a href="https://www.reddit.com/r/golang/comments/pi97sp/what_is_the_consequence_of_using_cgo_enabled0/">known issue</a> of Alpine image running a Go binary file, I followed the post and disabled <code>CGO</code> on SA, then <code>docker cp</code> the binary to CA, and it worked.</li>
</ol>
</div>
</div>
<div id="outline-container-org0423d38" class="outline-2">
<h2 id="org0423d38"><span class="section-number-2">3.</span> Another issue of running RPC in docker</h2>
<div class="outline-text-2" id="text-3">
<ul class="org-ul">
<li>The other day, I also spent hours debugging a <code>route unreachable</code> error when I want to send a request from CA to SA.</li>
<li>The CA is using the <code>bridge</code> network, so it should talk to SA via SA’s interface <code>docker0</code> within the subnet <code>172.17.0.0/16</code>.</li>
<li><p>
However, in my case, the docker by default rejects packages from any container as shown in SA’s <code>tcpdump</code> result:
</p>
<p class="verse">
172.17.0.1->172.17.0.3 ICMP host unreachable- admin prohibited, length 68<br>
</p></li>
<li><p>
By checking SA’s iptables, I found this rule:
</p>
<div class="org-src-container">
<pre class="src src-bash"> -A INPUT -j REJECT --reject-with icmp-host-prohibited
</pre>
</div>
<ul class="org-ul">
<li>Strangely, the <code>ping</code> still works with this rule.</li>
</ul></li>
<li><p>
In the end, I need to append a new rule to make the RPC work.
</p>
<div class="org-src-container">
<pre class="src src-bash"> iptables -I INPUT <span style="color: #da8548; font-weight: bold;">1</span> -i docker0 -p tcp --dport <port> -s 172.17.0.0/16 -j ACCEPT
</pre>
</div></li>
</ul>
</div>
</div>
<div class="taglist"><a href="https://chenyo-17.github.io/org-static-blog/tags.html">Tags</a>: <a href="https://chenyo-17.github.io/org-static-blog/tag-tools:docker.html">tools:docker</a> <a href="https://chenyo-17.github.io/org-static-blog/tag-lang:go.html">lang:go</a> <a href="https://chenyo-17.github.io/org-static-blog/tag-tools:linux:apline.html">tools:linux:apline</a> </div><div id="archive">
<a href="https://chenyo-17.github.io/org-static-blog/archive.html">Other posts</a>
</div>
</div>
<div id="postamble" class="status"></div>
</body>
</html>