Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test also inflection, wildcard, exact, compounded words #16

Open
wants to merge 156 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
156 commits
Select commit Hold shift + click to select a range
004247a
Test also inflection, wildcard, exact, compounded words
ksuess Mar 17, 2024
cac6840
lint
ksuess Mar 17, 2024
dfd48d1
lint now with correct image plone/frontend-dev with Node 20
ksuess Mar 17, 2024
93ca69a
Remove fields: index server address and index: block requests do use …
ksuess Mar 19, 2024
0a48418
Create Dockerfile.dev
ksuess Mar 21, 2024
da488be
pre-weekend-commit dev docker setup
ksuess Mar 22, 2024
034a5a4
Fix Sidebar
ksuess Mar 24, 2024
3e97ffa
format
ksuess Mar 24, 2024
ce96bf8
Fix CI code, unit
ksuess Mar 24, 2024
fb038e8
Use SSL
ksuess Mar 26, 2024
ca5fb2c
Extend tests
ksuess Mar 26, 2024
70358bd
Fix acceptance setup
ksuess Mar 26, 2024
8bf05cf
Fix preprocessing for blocks_plaintext
ksuess Mar 26, 2024
d1652f1
Fix dev setup
ksuess Mar 26, 2024
e699bea
Add blocks_plaintext to default searched fields
ksuess Mar 26, 2024
a777846
Test Opensearch SSL False
ksuess Mar 26, 2024
e7b337f
Fix field type of review_state to keyword. Fix blocks_plaintext
ksuess Apr 2, 2024
7005f9b
Fix mapping of "subjects"
ksuess Apr 2, 2024
58377e9
docker dev: Do not delete existing Plone site
ksuess Apr 2, 2024
97e6a13
Add pure start development/demo
ksuess Apr 2, 2024
0941df5
Back to current behavior: "@components/blocks_plaintext"
ksuess Apr 3, 2024
5c399c2
Update keywords and lexicon for text analysis
ksuess Apr 3, 2024
61a90f1
cypress test 'I can search for a compounded word'
ksuess Apr 3, 2024
a0bae1b
Workarround authentication bug in cypress tests
ksuess Apr 5, 2024
1c04d03
Remove dependency of acceptance containers for dev with backend add-ons
ksuess Apr 5, 2024
4b041b9
Reorder preprocessings.json
ksuess Apr 5, 2024
3bb6cec
Comment out elastic_search_api_url and index
ksuess Apr 5, 2024
b57ca9c
Label for facet and meta data item
ksuess Apr 5, 2024
551130e
Prepare multilingual tests
ksuess Apr 10, 2024
c282951
cypress configuration
ksuess Apr 10, 2024
924875d
Update ci.yml: depends_on
ksuess Apr 10, 2024
16418f9
Create search.multilingual.cy.js
ksuess Apr 10, 2024
40bfd01
Fix typo
ksuess Apr 10, 2024
d544a52
Add multilingual container
ksuess Apr 10, 2024
7181cca
Update Makefile
ksuess Apr 10, 2024
a60c62c
React components: First working step with language
ksuess Apr 10, 2024
6318db3
Include language in request only if site isMultilingual
ksuess Apr 10, 2024
eccaed2
Reduce default cypress configuration
ksuess Apr 11, 2024
b65b2c0
Uncomment single language acceptance tests
ksuess Apr 11, 2024
61fd962
Loosen cypress default specPattern
ksuess Apr 11, 2024
4aa8772
Explicit singlelingual cypress configuration
ksuess Apr 11, 2024
5d20deb
Temporary change from 'de' to 'it' until plone.volto:multilingual is …
ksuess Apr 11, 2024
6aa7fc5
Update Dockerfile.acceptance.multilingual
ksuess Apr 11, 2024
ea7f21f
Fix CI multilingual
ksuess Apr 11, 2024
38a5224
Fix dev acceptance multilingual
ksuess Apr 11, 2024
16d2b4d
Uncomment CI single language tests
ksuess Apr 11, 2024
165c5b6
Extend multilingual tests
ksuess Apr 11, 2024
4b82100
Clean up docker compose
ksuess Apr 11, 2024
516353c
Clean up
ksuess Apr 11, 2024
1ece406
Multilingual: Add analyzer. Mappings with fields. Add lexicon, keywords
ksuess Apr 11, 2024
d689846
Request with fields according language (z.B. "title.de_exact^1.4")
ksuess Apr 11, 2024
9f5fa25
Test multilingual first
ksuess Apr 11, 2024
7f118d4
Fix docker compose
ksuess Apr 12, 2024
186d1af
Search in language fields, but do filter on language only if isMultil…
ksuess Apr 12, 2024
979cd92
Fixi
ksuess Apr 20, 2024
8c33e88
Increase version (already released 0.4.0)
ksuess Jun 7, 2024
afceb49
Remove redundancy
ksuess Jun 7, 2024
e2041fe
import { createPortal } from 'react-dom';
ksuess Jun 7, 2024
22ea139
Delete .project.eslintrc.js
ksuess Jun 7, 2024
11fb0af
Remove dependencies (moved to @plone/volto)
ksuess Jun 7, 2024
0dd45a1
Revert "Delete .project.eslintrc.js"
ksuess Jun 7, 2024
dd5eb22
Update .project.eslintrc.js
ksuess Jun 8, 2024
86fa7f7
Update package.json
ksuess Jun 8, 2024
e996a6f
Delete webpack.config.js
ksuess Jun 8, 2024
eb8c4c9
Update .gitignore
ksuess Jun 16, 2024
32d4f26
Show dates (optional). Fix Test panel.
ksuess Jun 16, 2024
39a547c
make opensearchandingest (containers)
ksuess Jun 16, 2024
95bc263
Create dockerfiles/_env
ksuess Jun 16, 2024
1783b22
Update README.md
ksuess Jun 16, 2024
cdb55d0
Update ci.yml
ksuess Jun 16, 2024
461dc64
Update ci.yml
ksuess Jun 16, 2024
5f30053
naming: monolingual
ksuess Jun 16, 2024
d64c2b7
ci tests: first monolingual, then multilingual
ksuess Jun 16, 2024
903aa8e
More TODOs
ksuess Jun 16, 2024
5b6e97e
default for new fields
ksuess Jun 16, 2024
75e91be
Fix ci: either monolingual ingest or multilingual
ksuess Jun 16, 2024
61a95e8
Update yarn.lock
ksuess Jun 16, 2024
7d98eff
TO BE REVERTED! Temporary testing with unreleased collective.elastic.…
ksuess Jun 16, 2024
e4f06bb
Update Dockerfile.acceptance
ksuess Jun 16, 2024
2ee9962
Add TODOs
ksuess Jun 16, 2024
9e7c66b
clean up docker-compose acceptance
ksuess Jun 16, 2024
25d7088
clean up docker-compose CI
ksuess Jun 16, 2024
531ae9b
Update create_search.monolingual.cy.js
ksuess Jun 16, 2024
3c19ef4
Update ci.yml
ksuess Jun 16, 2024
df36c3d
Update Dockerfile.acceptance
ksuess Jun 16, 2024
61d93dc
Update Dockerfile.acceptance
ksuess Jun 16, 2024
f1b114f
Update ci.yml Explicitly export port of frontend
ksuess Jun 17, 2024
1b87e16
Update Dockerfile.acceptance.multilingual
ksuess Jun 17, 2024
3dd350e
Fix cypress multilingual
ksuess Jun 17, 2024
5505df1
Update language.multilingual.cy.js
ksuess Jun 17, 2024
61834ad
Update acceptance.yml: Split into two jobs
ksuess Jun 17, 2024
f6aa4c4
Update ci.yml
ksuess Jun 18, 2024
4ac9f67
Fix tests
ksuess Jun 18, 2024
c5f3cab
Get testing profiles from rohberg.volto
ksuess Jun 18, 2024
7bc69b5
Update language.multilingual.cy.js
ksuess Jun 18, 2024
c363bbb
Get testing profiles from rohberg.volto
ksuess Jun 18, 2024
bc4e78e
Fix tests
ksuess Jun 18, 2024
8b77baa
Volto 18
ksuess Jun 18, 2024
ffeb224
Add missing arg rohberg.volto
ksuess Jun 18, 2024
2648155
set build seed backend
ksuess Jun 18, 2024
d56ac87
yarn -> pnpm. We are testing with Volt 18
ksuess Jun 18, 2024
08bf8a1
Revert "yarn -> pnpm. We are testing with Volt 18"
ksuess Jun 18, 2024
f949bc0
Revert "Volto 18"
ksuess Jun 18, 2024
626293b
Back to Volto 17
ksuess Jun 18, 2024
994a058
Add missing arg rohberg.volto
ksuess Jun 18, 2024
8ce0d09
Update docker-compose.yml
ksuess Jun 18, 2024
263abb7
apply collective.elastic.plone:default
ksuess Jun 18, 2024
00d36f1
Add cypress aliases
ksuess Jun 18, 2024
85aef66
Add missing cypress alias
ksuess Jun 18, 2024
5029ec6
Add wait after creating content. Give ingest time to ingest
ksuess Jun 18, 2024
0164d9b
Update .env celery log level debug
ksuess Jun 19, 2024
d17a125
Fix typo CELERY_LOG_LEVEL
ksuess Jun 19, 2024
2933406
Switch to collective.elastic.plone[redis,opensearch] from repo main b…
ksuess Jun 19, 2024
a84e378
warning about missing licence
ksuess Jun 19, 2024
a4fbdb9
Fix locked search page
ksuess Jun 19, 2024
ab4606d
Fix typo CELERY_LOGLEVEL
ksuess Jun 19, 2024
a7add6c
wait for search results
ksuess Jun 19, 2024
e8272cc
Cypress testing: More waits
ksuess Jun 19, 2024
747ded5
Renaming tests
ksuess Jun 19, 2024
ac6edb2
cy.addNewBlock. Remove waits. cy.navigate
ksuess Jun 19, 2024
9bdd5af
CYPRESS_RETRIES: 3
ksuess Jun 19, 2024
79e05de
Remove clear before type
ksuess Jun 19, 2024
daca3f2
secrets.GITHUB_TOKEN
ksuess Jun 19, 2024
bcb16ba
defaultCommandTimeout: 8000
ksuess Jun 19, 2024
9588fd9
increase timout
ksuess Jun 19, 2024
979e67c
Update search.monolingual.cy.js
ksuess Jun 19, 2024
bfc1fbe
cy.navigate -> cy.visit
ksuess Jun 19, 2024
cf74119
Check publishing date. Check start date of event
ksuess Jun 19, 2024
08c257a
Fix eslint
ksuess Jun 23, 2024
8479c8b
locales
ksuess Jun 23, 2024
db0b61f
Fix flattenESUrlToPath without backend_url from block data
ksuess Jun 23, 2024
de32360
Move messages
ksuess Jun 23, 2024
21597dc
Easier configuration of block: no backend_url anymore
ksuess Jun 23, 2024
a494447
Update Results.js
ksuess Jun 23, 2024
50b2575
Fix jumping of page when navigating on search page
ksuess Jun 23, 2024
a01311a
No more backend_url from block data. Refactoring translate function /…
ksuess Jun 23, 2024
c042122
Update springisnow-volto-searchkit-block.less
ksuess Jun 23, 2024
5367415
Update index.js
ksuess Jun 23, 2024
abe5ed8
UI test panel
ksuess Jun 23, 2024
049ce1a
UI test panel #2
ksuess Jun 23, 2024
64e08fe
fieldset with name 'default' is required
ksuess Jun 23, 2024
f958729
Add test 'I can open a result'
ksuess Jun 23, 2024
cc5580b
Fix flattenESUrlToPath for multiple scenarios where everthing is runn…
ksuess Jun 23, 2024
818625b
Show head_title in results if available
ksuess Jun 23, 2024
64444c0
Test with Volto 17 and 18alpha
ksuess Jun 23, 2024
6315a51
Update acceptance.yml
ksuess Jun 23, 2024
8dff275
Update helpers.test.js
ksuess Jun 23, 2024
4ee1fa6
Update acceptance.yml
ksuess Jun 23, 2024
194b65b
Update helpers.test.js
ksuess Jun 23, 2024
e9f72f7
Update unit.yml matrix Volto
ksuess Jun 23, 2024
d35490d
Update unit.yml
ksuess Jun 23, 2024
7ab6ac7
Update unit.yml
ksuess Jun 23, 2024
1643b29
Update unit.yml
ksuess Jun 23, 2024
9e07c4c
Update acceptance.yml (test with Volto 18 postponed: pnpm, new projec…
ksuess Jun 23, 2024
9b35e1e
Update acceptance.yml
ksuess Jun 23, 2024
abd7f48
Update acceptance.yml
ksuess Jun 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
const fs = require('fs');
const projectRootPath = __dirname;

let coreLocation;
if (fs.existsSync(`${projectRootPath}/core`))
coreLocation = `${projectRootPath}/core`;
else if (fs.existsSync(`${projectRootPath}/../../core`))
coreLocation = `${projectRootPath}/../../core`;

module.exports = {
extends: `${coreLocation}/packages/volto/.eslintrc`,
rules: {
'import/no-unresolved': 1,
},
settings: {
'import/resolver': {
alias: {
map: [
['@plone/volto', `${coreLocation}/packages/volto/src`],
[
'@plone/volto-slate',
`${coreLocation}/core/packages/volto-slate/src`,
],
['@plone/registry', `${coreLocation}/packages/registry/src`],
['@rohberg/volto-searchkit-block', './src'],
],
extensions: ['.js', '.jsx', '.ts', '.tsx', '.json'],
},
},
},
};
77 changes: 68 additions & 9 deletions .github/workflows/acceptance.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,18 @@ name: Acceptance tests
on: [push]

env:
ADDON_NAME: "@rohberg/volto-searchkit-block"
ADDON_NAME: '@rohberg/volto-searchkit-block'
ADDON_PATH: volto-searchkit-block
VOLTO_VERSION: 17
PLONE_VERSION: 6.0

INDEX_SERVER: opensearch:9200
INDEX_OPENSEARCH: 1
INDEX_USE_SSL: 0
INDEX_LOGIN: admin
INDEX_PASSWORD: "oxczBG).3xWyapLn"
INDEX_PASSWORD: 'oxczBG).3xWyapLn'

CELERY_BROKER: redis://redis:6379/0
CELERY_LOG_LEVEL: info
CELERY_LOGLEVEL: info

PLONE_SERVICE: http://backend-acceptance:55001
PLONE_SITE_PREFIX_PATH: plone
Expand All @@ -26,11 +25,13 @@ env:
PREPROCESSINGS_FILE: /configuration/preprocessings.json

jobs:
acceptance:
acceptance-monolingual:
runs-on: ubuntu-latest
timeout-minutes: 45
timeout-minutes: 46
strategy:
fail-fast: false
matrix:
volto-version: [17]
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -43,16 +44,21 @@ jobs:
cd acceptance
yarn

- name: "Cypress: Acceptance tests"
- name: 'Cypress: Acceptance tests - monolingual'
uses: cypress-io/github-action@v6
env:
VOLTO_VERSION: ${{ matrix.volto-version }}
BABEL_ENV: production
CYPRESS_RETRIES: 2
CYPRESS_RETRIES: 3
# Recommended: pass the GitHub token lets this action correctly
# determine the unique run id necessary to re-run the checks
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
config: pageLoadTimeout=100000,defaultCommandTimeout=8000
parallel: false
browser: chrome
working-directory: acceptance
spec: cypress/tests/*.cy.js
spec: cypress/tests/*.monolingual.cy.js
install: false
start: |
docker compose -f ci.yml --profile prod up
Expand All @@ -71,3 +77,56 @@ jobs:
with:
name: cypress-videos-acceptance
path: acceptance/cypress/videos

acceptance-multilingual:
runs-on: ubuntu-latest
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
volto-version: [17]
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Install Cypress
run: |
cd acceptance
yarn

- name: 'Cypress: Acceptance tests - multilingual'
uses: cypress-io/github-action@v6
env:
VOLTO_VERSION: ${{ matrix.volto-version }}
BABEL_ENV: production
CYPRESS_RETRIES: 3
# Recommended: pass the GitHub token lets this action correctly
# determine the unique run id necessary to re-run the checks
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
config: pageLoadTimeout=100000,defaultCommandTimeout=8000
parallel: false
browser: chrome
working-directory: acceptance
spec: cypress/tests/*.multilingual.cy.js
install: false
start: |
docker compose -f ci.yml --profile multilingual up
wait-on: 'npx wait-on --httpTimeout 20000 http-get://localhost:55001/plone http://localhost:3000 http://localhost:9200'

# Upload Cypress screenshots
- uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-screenshots-acceptance
path: acceptance/cypress/screenshots

# Upload Cypress videos
- uses: actions/upload-artifact@v4
if: failure()
with:
name: cypress-videos-acceptance
path: acceptance/cypress/videos
25 changes: 25 additions & 0 deletions .github/workflows/code.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,30 @@
name: Code analysis checks
on: [push]

env:
ADDON_NAME: '@rohberg/volto-searchkit-block'
ADDON_PATH: volto-searchkit-block
VOLTO_VERSION: 17
PLONE_VERSION: 6.0

INDEX_SERVER: opensearch:9200
INDEX_OPENSEARCH: 1
INDEX_USE_SSL: 0
INDEX_LOGIN: admin
INDEX_PASSWORD: 'oxczBG).3xWyapLn'

CELERY_BROKER: redis://redis:6379/0
CELERY_LOGLEVEL: info

PLONE_SERVICE: http://backend-acceptance:55001
PLONE_SITE_PREFIX_PATH: plone
PLONE_USER: admin
PLONE_PASSWORD: secret

MAPPINGS_FILE: /configuration/mappings.json
ANALYSIS_FILE: /configuration/analysis.json
PREPROCESSINGS_FILE: /configuration/preprocessings.json

jobs:
codeanalysis:
runs-on: ubuntu-latest
Expand Down
28 changes: 28 additions & 0 deletions .github/workflows/unit.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,43 @@
name: Unit Tests
on: [push]

env:
ADDON_NAME: '@rohberg/volto-searchkit-block'
ADDON_PATH: volto-searchkit-block
VOLTO_VERSION: 17
PLONE_VERSION: 6.0

INDEX_SERVER: opensearch:9200
INDEX_OPENSEARCH: 1
INDEX_USE_SSL: 0
INDEX_LOGIN: admin
INDEX_PASSWORD: 'oxczBG).3xWyapLn'

CELERY_BROKER: redis://redis:6379/0
CELERY_LOGLEVEL: info

PLONE_SERVICE: http://backend-acceptance:55001
PLONE_SITE_PREFIX_PATH: plone
PLONE_USER: admin
PLONE_PASSWORD: secret

MAPPINGS_FILE: /configuration/mappings.json
ANALYSIS_FILE: /configuration/analysis.json
PREPROCESSINGS_FILE: /configuration/preprocessings.json

jobs:
unit:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [20.x]
volto-version: [17]

steps:
- name: Main checkout
uses: actions/checkout@v4

- name: Unit tests
# TODO do 'docker compose with ' env: VOLTO_VERSION: ${{ matrix.volto-version }}
# TODO Remove matrix option node-version
run: make test-ci
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ api/constraints-mxdev.txt
api/requirements-mxdev.txt
development-searchkitblock/celery/venv/
development-searchkitblock/celery/sources/
acceptance/cypress/screenshots/
build/
.yarn/
yarn.lock

!docker-opensearch/.env
!dockerfiles/opensearch/.env
!acceptance/.env
50 changes: 0 additions & 50 deletions .project.eslintrc.js

This file was deleted.

43 changes: 18 additions & 25 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,40 @@
# Development
# Development and demo

## Backend
> TODO Update to new Makefile

make dev-start-backend

## Elasticsearch
## Index server OpenSearch

Run with Docker.
make dev-opensearch

Change directory to ./development-searchkitblock/mac/ or ./development-searchkitblock/linux/ and run:
## Backend and frontend

docker compose up
make dev

Inspect with

docker exec -it <container-id> bash

## Redis

Start with

redis-server /usr/local/etc/redis.conf
> OUTDATED OLD STUFF BELOW

## Celery
## Backend

Change directory to ./development-searchkitblock/celery and install:
make dev-start-backend

Install with:

python -m venv venv
source venv/bin/activate
pip install -U pip wheel mxdev
mxdev -c mx.ini
## OUTDATED Elasticsearch

pip install -r requirements-mxdev.txt
Run with Docker.

Change directory to ./development-searchkitblock/mac/ or ./development-searchkitblock/linux/ and run:

docker compose up

Inspect with

Run with:
docker exec -it <container-id> bash

source .env
venv/bin/celery -A collective.elastic.ingest.celery.app worker -l info

# or with more info:
venv/bin/celery -A collective.elastic.ingest.celery.app worker -l debug


## Frontend
Expand Down
Loading