Enable use in addons

[rodfersou]

The idea is to have a docker-compose.yml file in one addon like this:

version: '2'
services:
  plone:
    container_name: plone
    image: plone:5.2.1
    volumes:
      - .:/app

And run buildout and instance fg from this image using the already present cache

[avoinea]

@rodfersou I think you can already do that like:

docker-compose.yml

version: "2"
services:
  plone:
    image: plone:5
    ports:
    - "8080:8080"
    environment:
     ADDONS=my.addon
     DEVELOP=src/my.addon
   volumes:
   - ./:/plone/instance/src/my.addon
   command:
   - fg

See https://github.com/plone/plone.docker/blob/master/docs/usage/addons.rst#add-ons

[avoinea]

We intentionally removed gcc/make from this image in order to make it more lighter and secure.

99% of the cases you shouldn't need to build libraries from source code within container. For the remaining of 1% we encourage people to create custom Docker images based on this one and do the compilation at the image level.

[avoinea]

This shouldn't be needed as it's the default place where downloads and eggs are stored.

[rodfersou]

@avoinea understand, thanks for the explanation.

by the way, the docker-compose suggestion didn't work (I fixed some indentation to actually run):

version: '2'
services:
  plone:
    image: plone:5.2.1
    ports:
    - '8080:8080'
    environment:
    - ADDONS=collective.taxonomy
    - DEVELOP=src/collective.taxonomy
    volumes:
    - .:/plone/instance/src/collective.taxonomy
    command:
    - fg

And this was the error I got:

$ docker-compose up
Creating network "collectivetaxonomy_default" with the default driver
Creating collectivetaxonomy_plone_1 ... done
Attaching to collectivetaxonomy_plone_1
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/include/python3.7m': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/shutil.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/tempfile.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/lib-dynload': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/encodings': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/_dummy_thread.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/copyreg.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/rlcompleter.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/token.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/base64.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/sre_constants.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/weakref.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/bisect.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/heapq.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/functools.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/hashlib.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/fnmatch.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/warnings.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/codecs.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/_weakrefset.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/io.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/operator.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/os.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/copy.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/genericpath.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/linecache.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/types.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/random.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/keyword.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/reprlib.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/readline.so': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/posixpath.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/stat.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/config-3.7m-darwin': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/sre_compile.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/locale.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/_collections_abc.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/__future__.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/enum.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/hmac.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/tarfile.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/importlib': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/LICENSE.txt': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/collections': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/ntpath.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/tokenize.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/imp.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/re.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/struct.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/sre_parse.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/abc.py': No such file or directory
plone_1  | chown: cannot dereference '/plone/instance/src/collective.taxonomy/lib/python3.7/_bootlocale.py': No such file or directory
collectivetaxonomy_plone_1 exited with code 1

I'll probably look for a custom image in my next try! thanks

[avoinea]

by the way, the docker-compose suggestion didn't work (I fixed some indentation to actually run):

@rodfersou

It will work on a fresh clone. In your case it didn't work because you also ran the buildout inside that egg and within docker container it's trying to resolve some symlinks from lib/python3.7

[rodfersou]

understand

[rodfersou]

@avoinea I still think would be better if this package have all we need as environment (tools and cache) to build and start a Plone instance, not to include Plone itself inside.

My workflow would be to mount my directory and say "hey docker, build this package for me" then "hey docker, start my instance".

That's why I think it is necessary to add make and properly configure the cache.
I want to test my buildout, not to use my addon inside the default buildout 😄

Do you think we can get a midterm between those two worlds? Where we use today those docker images?

[avoinea]

@rodfersou I see.

I'll take it into consideration for 5.2.2 release 😉

[svx]

@avoinea I still think would be better if this package have all we need as environment (tools and cache) to build and start a Plone instance, not to include Plone itself inside.

My workflow would be to mount my directory and say "hey docker, build this package for me" then "hey docker, start my instance".

That's why I think it is necessary to add make and properly configure the cache.
I want to test my buildout, not to use my addon inside the default buildout 😄

Do you think we can get a midterm between those two worlds? Where we use today those docker images?

IMHO this not really, cool :) Adding more and more stuff make the images bigger, slower to build, pull and add more security concerns.

If you want something like that, than IMHO you/the community could/should switch to multi stage build images, with that you can have a minimal one and use that to build an extend one on top of that.
As community we could even release multiple oficial ones, think of it like you have for example with Python, we could have "fat" images with all the compilers and "small" ones.

One other important point to consider, the current images may already used by people in prod (even if we do not advice that) by adding now suddenly compilers and other things you could compromise their flow/pipline/setup.

So that means you should build and tag them different anyway 😄

[avoinea]

One other important point to consider, the current images may already used by people in prod (even if we do not advice that) by adding now suddenly compilers and other things you could compromise their flow/pipline/setup.

So that means you should build and tag them different anyway

Yes, we can consider it for the next release.

I don't think that having a default buildout configuration for the plone user will do any harm.

About having make inside, we can think about it:

debian

Reading state information... Done
Suggested packages:
  make-doc
The following NEW packages will be installed:
  make
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 302 kB of archives.
After this operation, 1240 kB of additional disk space will be used.

alpine

bash-5.0# du -sh /
523.6M	/
bash-5.0# apk add --no-cache make
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.12/community/x86_64/APKINDEX.tar.gz
(1/1) Installing make (4.3-r0)
Executing busybox-1.31.1-r16.trigger
bash-5.0# du -sh /
523.8M	/

It has not that much impact on the image size (not sure about security, though), and the benefit is that you will be able to easily run complex commands from a Makefile like:

make code-format-backend
make test-backend

[svx]

For me it is also thinking "long term" now we may add make next month maybe another package and so on and so on :)

That is why I suggest moving to multi stage and creating a "fat" and a "slim" version.

Also, if people really want to add more packages/functions they can always use the "official" image as base for building their own one. :)

What I am trying to say is, I want to avoid opening a can of worms, here. As more people start using the images more people want to have more "default included things" because that is what they need/want for their use case :)

[avoinea]

😄 Then we can go for multi stage/multi tags.

Now that Python2 is gone, we have space to explore more tags, like for example:

• RelStorage
• pip install Plone
• RestAPI only
• dev
• etc.

[rodfersou]

@svx I think we underestimate the use of Docker, it is meant to have same execution environment for development, CI, staging and production, but the way it is now it just can't fit in the buildout we use today.

For addon development, what I act more today, from time to time we suffer trying to reproduce the same enviroment from Travis or Jenkins, because locally it just works, but tests don't pass.

Make is being used more and more for addons at the community, so it is worth to have it in Docker.

My target here is to don't need anymore to install a server or development environment from ground up, just use the Docker image.

I do care about keep it small (for me the default buildout should not be inside the image, we can just clone outside and run).

But of course, if those small changes are not good for the official image I'll go to create a new unofficial one 😄

[rodfersou]

@avoinea I could make a setup based on current image collective/collective.taxonomy#102

in this case I need to install also virtualenv

next step would be to try to use docker on travis I think.

[avoinea]

@rodfersou As I see you already fixed your issue directly in collective.taxonomy I'll close this and skip it for now.

[rodfersou]

although it was not a good fix 😂