Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

change the luasocket feature to haproxy sockets #1

Closed
wants to merge 23 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
138 changes: 87 additions & 51 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,41 +2,53 @@

![test](https://github.com/leafo/pgmoon/workflows/test/badge.svg)

> **Note!** Are you using the latest version of OpenResty? You must update to
> pgmoon 1.12 or above, due to a change in Lua pattern compatibility, any query
> that returns affected number of rows will return the expected value.
> **Note:** Have you updated from an older version of OpenResty? You must update to
> pgmoon 1.12 or above, due to a change in Lua pattern compatibility to avoid incorrect
> results from queries that return affected rows.

pgmoon is a PostgreSQL client library written in pure Lua (MoonScript).
**pgmoon** is a PostgreSQL client library written in pure Lua (MoonScript).

pgmoon was originally designed for use in [OpenResty][5] to take advantage of
the [cosocket api][4] to provide asynchronous queries but it also works in the
regular Lua environment as well using [LuaSocket][1] (and optionally
[luaossl](https://luarocks.org/modules/daurnimator/luaossl) or [LuaCrypto][2] for MD5 authentication and [LuaSec][6] for SSL connections).
pgmoon can also use [cqueues][]' socket when passed `"cqueues"` as the socket
type when instantiating.
**pgmoon** was originally designed for use in [OpenResty][] to take advantage
of the [cosocket
api](https://github.com/openresty/lua-nginx-module#ngxsockettcp) to provide
asynchronous queries but it also works in the regular any Lua environment where
[LuaSocket][] or [cqueues][] is available.

It's a perfect candidate for running your queries both inside OpenResty's
environment and on the command line (eg. tests) in web frameworks like [Lapis][3].
environment and on the command line (eg. tests) in web frameworks like [Lapis][].

## Install

```bash
$ luarocks install pgmoon
```

<details>
<summary>Using <a href="https://opm.openresty.org/">OpenResty's OPM</a></summary>

```bash
$ opm get leafo/pgmoon
```

</details>


### Dependencies

pgmoon supports a wide range of environments and libraries, so it may be
necessary to install additional dependencies depending on how you intend to
communicate with the database:

> Tip: If you're using OpenResty then no additional dependencies are needed
> **Tip:** If you're using OpenResty then no additional dependencies are needed
> (generally, a crypto library may be necessary for some authentication
> methods)

Some socket library **is required**, depending on the environment you can chose one:
A socket implementation **is required** to use pgmoon, depending on the
environment you can chose one:

* [OpenResty](https://openresty.org/en/) &mdash; The built in socket is used, so additional dependencies necessary
* [LuaSocket](http://w3.impa.br/~diego/software/luasocket/) &mdash; Suitable for command line database access, has the highest platform compatibility `luarocks install luasocket`
* [cqueues](https://github.com/wahern/cqueues) &mdash; `luarocks install cqueues`
* [OpenResty][] &mdash; The built in socket is used, no additional dependencies necessary
* [LuaSocket][] &mdash; `luarocks install luasocket`
* [cqueues][] &mdash; `luarocks install cqueues`

If you're on PUC Lua 5.1 or 5.2 then you will need a bit libray (not needed for LuaJIT):

Expand All @@ -50,19 +62,21 @@ If you want to use JSON types you will need lua-cjson
$ luarocks install lua-cjson
```

If you want to use SSL connections with LuaSocket then you will need LuaSec:
(OpenResty and cqueues come with their own SSL implementations)
SSL connections may require an additional dependency:

* OpenResty &mdash; No additional dependencies required
* LuaSocket &mdash; `luarocks install luasec`
* cqueues &mdash; `luarocks install luaossl`

Password authentication may require a crypto library, [luaossl][].

```bash
$ luarocks install luasec
$ luarocks install luaossl
```

If you want to use password authentication then you will need a crypto library:
> **Note:** [LuaCrypto][] can be used as a fallback, but the library is abandoned and not recommended for use

* [OpenResty](https://openresty.org/en/) &mdash; Built in function will be used, no additional dependencies necessary
* [luaossl](https://github.com/wahern/luaossl) &mdash; Recommended `luarocks install luaossl`
* [luacrypto](https://github.com/starius/luacrypto) &mdash; Deprecated library, not recommended
> **Note:** Use within [OpenResty][] will prioritize built in functions if possible

## Example

Expand All @@ -81,8 +95,8 @@ local res = assert(pg:query("select * from users where username = " ..
pg:escape_literal("leafo")))
```

If you are using OpenResty you should relinquish the socket after you are done
with it so it can be reused in future requests:
If you are using OpenResty you can relinquish the socket to the connection pool
after you are done with it so it can be reused in future requests:

```lua
pg:keepalive()
Expand All @@ -94,19 +108,27 @@ Functions in table returned by `require("pgmoon")`:

### `new(options={})`

Creates a new `Postgres` object. Does not connect automatically. Takes a table
of options. The table can have the following keys:
Creates a new `Postgres` object from a configuration object. All fields are
optional unless otherwise stated. The newly created object will not
automatically connect, you must call `conect` after creating the object.

Available options:

* `"database"`: the database name to connect to **required**
* `"host"`: the host to connect to (default: `"127.0.0.1"`)
* `"port"`: the port to connect to (default: `"5432"`)
* `"user"`: the database username to authenticate (default: `"postgres"`)
* `"database"`: the database name to connect to **required**
* `"password"`: password for authentication, optional depending on server configuration
* `"password"`: password for authentication, may be required depending on server configuration
* `"ssl"`: enable ssl (default: `false`)
* `"ssl_verify"`: verify server certificate (default: `nil`)
* `"ssl_required"`: abort the connection if the server does not support SSL connections (default: `nil`)
* `"pool"`: optional name of pool to use when using OpenResty cosocket (defaults to `"#{host}:#{port}:#{database}"`)
* `"socket_type"`: optional, the type of socket to use, one of: `"nginx"`, `"luasocket"`, `cqueues` (default: `"nginx"` if in nginx, `"luasocket"` otherwise)
* `"socket_type"`: the type of socket to use, one of: `"nginx"`, `"luasocket"`, `cqueues` (default: `"nginx"` if in nginx, `"luasocket"` otherwise)
* `"application_name"`: set the name of the connection as displayed in `pg_stat_activity`. (default: `"pgmoon"`)
* `"pool"`: (OpenResty only) name of pool to use when using OpenResty cosocket (default: `"#{host}:#{port}:#{database}"`)
* `"pool_size"`: (OpenResty only) Passed directly to OpenResty cosocket connect function, [see docs](https://github.com/openresty/lua-nginx-module#tcpsockconnect)
* `"backlog"`: (OpenResty only) Passed directly to OpenResty cosocket connect function, [see docs](https://github.com/openresty/lua-nginx-module#tcpsockconnect)
* `"cqueues_openssl_context"`: Manually created `opensssl.ssl.context` to use when created cqueues SSL connections
* `"luasec_opts"`: Manually created options object to use when using LuaSec SSL connections

Methods on the `Postgres` object returned by `new`:

Expand All @@ -119,9 +141,8 @@ message.

### postgres:settimeout(time)

Sets the timeout value (in milliseconds) for all socket operations (connect,
write, receive). This function does not have any return values.

Sets the timeout value (in milliseconds) for all subsequent socket operations
(connect, write, receive). This function does not have any return values.

### success, err = postgres:disconnect()

Expand Down Expand Up @@ -252,9 +273,9 @@ Returns string representation of current state of `Postgres` object.
## SSL connections

pgmoon can establish an SSL connection to a Postgres server. It can also refuse
to connect to it if the server does not support SSL.
Just as pgmoon depends on LuaSocket for usage outside of OpenResty, it depends
on LuaSec for SSL connections in such contexts.
to connect to it if the server does not support SSL. Just as pgmoon depends on
LuaSocket for usage outside of OpenResty, it depends on luaossl/LuaSec for SSL
connections in such contexts.

```lua
local pgmoon = require("pgmoon")
Expand All @@ -272,13 +293,13 @@ local pg = pgmoon.new({
assert(pg:connect())
```

> Note: In Postgres 12 and above, the minium SSL version accepted by client
> connections is 1.2. When using LuaSec to connect to an SSL server, if you
> don't specify an `ssl_version` then `tlsv1_2` is used.
> **Note:** In Postgres 12 and above, the minium SSL version accepted by client
> connections is 1.2. When using LuaSocket + LuaSec to connect to an SSL
> server, if you don't specify an `ssl_version` then `tlsv1_2` is used.

In OpenResty, make sure to configure the [lua_ssl_trusted_certificate][7]
directive if you wish to verify the server certificate, as the LuaSec-only
options become irrelevant in that case.
In OpenResty, make sure to configure the
[lua_ssl_trusted_certificate](https://github.com/openresty/lua-nginx-module#lua_ssl_trusted_certificate)
directive if you wish to verify the server certificate.

## Authentication types

Expand Down Expand Up @@ -319,6 +340,22 @@ local my_array = {1,2,3,4,5}
pg:query("insert into some_table (some_arr_col) values(" .. encode_array(my_array) .. ")")
```

### Empty Arrays

When trying to encode an empty array an error will be thrown. Postgres requires
a type when using an array. When there are values in the array Postgres can
infer the type, but with no values in the array no type can be inferred. This
is illustrated in the erorr provided by Postgres:


```
postgres=# select ARRAY[];
ERROR: cannot determine type of empty array
LINE 1: select ARRAY[];
^
HINT: Explicitly cast to the desired type, for example ARRAY[]::integer[].
```

## Handling JSON

`json` and `jsonb` types are automatically decoded when they are returned from
Expand Down Expand Up @@ -397,6 +434,7 @@ Homepage: <http://leafo.net>

# Changelog

* 1.13.0 — 2021-10-13 - Add support for scram_sha_256_auth (@murillopaula), 'backlog' and 'pool_size' options while using ngx.socket (@xiaocang), update LuaSec ssl_protocol default options (@jeremymv2), `application_name` option (@mecampbellsoup)
* 1.12.0 — 2021-01-06 - Lua pattern compatibility fix, Support for Lua 5.1 through 5.4 (@jprjr). Fix bug where SSL vesrion was not being passed. Default to TLS v1.2 when using LuaSec. Luabitop is no longer automatically installed as a dependency. New test suite.
* 1.11.0 — 2020-03-26 - Allow for TLS v1.2 when using LuaSec (Miles Elam)
* 1.10.0 — 2019-04-15 - Support luaossl for crypto functions, added better error when missing crypto library
Expand Down Expand Up @@ -434,12 +472,10 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.


[1]: http://w3.impa.br/~diego/software/luasocket/
[2]: http://mkottman.github.io/luacrypto/
[3]: http://leafo.net/lapis
[4]: http://wiki.nginx.org/HttpLuaModule#ngx.socket.tcp
[5]: http://openresty.org/
[6]: https://github.com/brunoos/luasec
[7]: https://github.com/openresty/lua-nginx-module#lua_ssl_trusted_certificate
[luaossl]: https://github.com/wahern/luaossl
[LuaCrypto]: https://luarocks.org/modules/starius/luacrypto
[LuaSec]: https://github.com/brunoos/luasec
[Lapis]: http://leafo.net/lapis
[OpenResty]: https://openresty.org/
[LuaSocket]: http://w3.impa.br/~diego/software/luasocket/
[cqueues]: http://25thandclement.com/~william/projects/cqueues.html
1 change: 0 additions & 1 deletion Tupfile

This file was deleted.

1 change: 0 additions & 1 deletion Tuprules.tup

This file was deleted.

3 changes: 2 additions & 1 deletion pgmoon-dev-1.rockspec
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@ package = "pgmoon"
version = "dev-1"

source = {
url = "git://github.com/leafo/pgmoon.git"
url = "git://github.com/kriscode1/pgmoon.git",
tag = "luasocket-to-haproxy"
}

description = {
Expand Down
21 changes: 15 additions & 6 deletions pgmoon/arrays.lua
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,25 @@ getmetatable(PostgresArray).__call = function(self, t)
return setmetatable(t, self.__base)
end
local default_escape_literal = nil
local insert, concat
do
local _obj_0 = table
insert, concat = _obj_0.insert, _obj_0.concat
end
local encode_array
do
local append_buffer
append_buffer = function(escape_literal, buffer, values)
for _index_0 = 1, #values do
local item = values[_index_0]
if type(item) == "table" and not getmetatable(item) then
table.insert(buffer, "[")
insert(buffer, "[")
append_buffer(escape_literal, buffer, item)
buffer[#buffer] = "]"
table.insert(buffer, ",")
insert(buffer, ",")
else
table.insert(buffer, escape_literal(item))
table.insert(buffer, ",")
insert(buffer, escape_literal(item))
insert(buffer, ",")
end
end
return buffer
Expand All @@ -53,8 +58,12 @@ do
local buffer = append_buffer(escape_literal, {
"ARRAY["
}, tbl)
buffer[#buffer] = "]"
return table.concat(buffer)
if buffer[#buffer] == "," then
buffer[#buffer] = "]"
else
insert(buffer, "]")
end
return concat(buffer)
end
end
local convert_values
Expand Down
18 changes: 12 additions & 6 deletions pgmoon/arrays.moon
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,20 @@ getmetatable(PostgresArray).__call = (t) =>

default_escape_literal = nil

import insert, concat from table

encode_array = do
append_buffer = (escape_literal, buffer, values) ->
for item in *values
-- plain array
if type(item) == "table" and not getmetatable(item)
table.insert buffer, "["
insert buffer, "["
append_buffer escape_literal, buffer, item
buffer[#buffer] = "]" -- strips trailing comma
table.insert buffer, ","
insert buffer, ","
else
table.insert buffer, escape_literal item
table.insert buffer, ","
insert buffer, escape_literal item
insert buffer, ","

buffer

Expand All @@ -33,8 +35,12 @@ encode_array = do

buffer = append_buffer escape_literal, {"ARRAY["}, tbl

buffer[#buffer] = "]" -- strips trailing comma
table.concat buffer

if buffer[#buffer] == ","
buffer[#buffer] = "]"
else
insert buffer, "]"
concat buffer

convert_values = (array, fn) ->
for idx, v in ipairs array
Expand Down
Loading