Skip to content

Commit

Permalink
[Utils] Share configuration between 3rd parties related scripts
Browse files Browse the repository at this point in the history
- Now the THIRD-PARTY generator has default values
- Prompt and generator scripts share configuration values

Signed-off-by: Pierre-Yves Lapersonne <[email protected]>
  • Loading branch information
pylapp committed Mar 20, 2024
1 parent d6a1ed2 commit c29a4bf
Show file tree
Hide file tree
Showing 5 changed files with 44 additions and 29 deletions.
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased](https://github.com/Orange-OpenSource/floss-toolbox/compare/2.16.0..dev)

### Changed

- [Utils] Default values for THIRD_PARTY generator script, shared configuration with prompt script

## [2.16.0](https://github.com/Orange-OpenSource/floss-toolbox/compare/2.16.0..2.15.0) - 2024-03-16

### Added
Expand Down
11 changes: 8 additions & 3 deletions toolbox/utils/third-party-generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,9 @@ so as to iterate on each component and build the final Markdown file.
# --delimiter: to define how to split each row fields. Do not forget to escape it if ';'
# --avoid: if a version or copyright field has "?" as value, do not add it in generated file
python3.8 third-party-generator.py --file components.csv.result --delimiter \; --avoid \?

# or use the default configuration
python3.8 third-party-generator.py
```

### About the CSV file
Expand All @@ -53,7 +56,7 @@ Meaning:
- version: the verison of the component


For example, with the CSV file bellow
For example, with the CSV file bellow (without the first line containing headers, do fill it)
```csv
SwiftUI-Flow;https://github.com/tevelee/SwiftUI-Flow;MIT;Copyright (c) 2023 Laszlo Teveli;1.2.0
BottomSheet;https://github.com/lucaszischka/BottomSheet;MIT;Copyright (c) 2021-2022 Lucas Zischka;3.1.1
Expand Down Expand Up @@ -99,8 +102,10 @@ Copyright (c) 2021-2023 Orange SA
You may download the source code on the [following website](https://github.com/Orange-OpenSource/accessibility-statement-lib-ios).
```

### About the licenses.py file
### About the configuration.py file

There is plenty of licenses and also a lot of standards. It can be a pain or time-consuming to let the user write the license in use for a component,
then find the URL pointing to the license text and write it. In fact, such details are still known so we can let the user choose within list items.
The *licenses.py* file lists main licenses we can meet during audits. Each entry in this dictionary has a license name in SPDX short-identifier format and the URL pointing to the license text. Thus these details will be added in the THIRD-PARTY file.
The *configuration.py* file lists main licenses we can meet during audits. Each entry in this dictionary has a license name in SPDX short-identifier format and the URL pointing to the license text. Thus these details will be added in the THIRD-PARTY file.

It contains also shared configuration between scripts.
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,20 @@
# Authors: See CONTRIBUTORS.txt
# Software description: A toolbox of scripts to help work of forges admins and open source referents

# Version.............: 1.0.0
# Version.............: 1.1.0
# Since...............: 12/03/2024
# Description.........: Just to list managed licenses and share values between scripts

# Some configuration
# ------------------

DEFAULT_PROMPT_RESULT_FILE = "components.csv.result"
DEFAULT_PROMPT_RESULT_FILE_DELIMITER = ";"
DEFAULT_AVOID_FIELD_SYMBOL = "?"

# Licenses to manage
# ------------------

# Please refer to https://opensource.org/licenses or https://spdx.org/licenses/
# Use SPDX short identifier as key, URL of license text as value
LICENSES = {
Expand Down
12 changes: 6 additions & 6 deletions toolbox/utils/third-party-generator/third-party-generator.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@
# Authors: See CONTRIBUTORS.txt
# Software description: A toolbox of scripts to help work of forges admins and open source referents

# Version.............: 2.0.0
# Version.............: 3.0.0
# Since...............: 12/03/2024
# Description.........: Builds a third-party Markdown based on a CSV file and a delimiter
# Description.........: Builds a third-party Markdown file based on a CSV file and a delimiter

import argparse
import csv
import os
import sys

from licenses import *
from configuration import *

# Configuration
# -------------
Expand All @@ -36,9 +36,9 @@

parser = argparse.ArgumentParser(description='This script will build a third-party file listing components based on a CSV file and a delimiter')
required_args = parser.add_argument_group('Required arguments')
required_args.add_argument('-f', '--file', help='The CSV file file to process', required=True)
required_args.add_argument('-d', '--delimiter', help='The delimter symbol (e.g. ";" to split fields for each line of the CSV file', required=True)
required_args.add_argument('-a', '--avoid', help='The sequence to use to define wether or not a specific field (version or copyright) must be ignored (e.g. "?" in the CSV raw field', required=True)
required_args.add_argument('-f', '--file', help='The CSV file file to process', default=DEFAULT_PROMPT_RESULT_FILE)
required_args.add_argument('-d', '--delimiter', help='The delimter symbol (e.g. ";" to split fields for each line of the CSV file', default=DEFAULT_PROMPT_RESULT_FILE_DELIMITER)
required_args.add_argument('-a', '--avoid', help='The sequence to use to define wether or not a specific field (version or copyright) must be ignored (e.g. "?" in the CSV raw field', default=DEFAULT_AVOID_FIELD_SYMBOL)
args = parser.parse_args()

content_file_name = args.file
Expand Down
34 changes: 15 additions & 19 deletions toolbox/utils/third-party-generator/third-party-prompt.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# Authors: See CONTRIBUTORS.txt
# Software description: A toolbox of scripts to help work of forges admins and open source referents

# Version.............: 1.1.0
# Version.............: 1.2.0
# Since...............: 12/03/2024
# Description.........: Builds a CSV file based on user inputs.

Expand All @@ -19,7 +19,7 @@
import sys

from collections import defaultdict
from licenses import *
from configuration import *

# Configuration
# -------------
Expand All @@ -28,10 +28,6 @@
EXIT_OK = 0
ERROR_BAD_ARGUMENTS = 1

# Other config
SAVE_FILE = "components.csv.result"
SAVE_FILE_DELIMITER = ";"

# Methods
# -------

Expand All @@ -44,8 +40,8 @@ def check_value(value):
if not value:
print("✋ Woops, value must be defined")
return False
if SAVE_FILE_DELIMITER in value:
print(f'✋ Data are exported in CSV, avoid using "{SAVE_FILE_DELIMITER}"')
if DEFAULT_PROMPT_RESULT_FILE_DELIMITER in value:
print(f'✋ Data are exported in CSV, avoid using "{DEFAULT_PROMPT_RESULT_FILE_DELIMITER}"')
return False
return True

Expand All @@ -62,20 +58,20 @@ def check_value(value):
sys.exit(ERROR_BAD_ARGUMENTS)

# Check if temporary file exists and ask to recover or delete it
if os.path.isfile(SAVE_FILE):
if os.path.isfile(DEFAULT_PROMPT_RESULT_FILE):
print("⚠️ A previous backup exists")
input_user_answer = input("❓ Do you want to use it? (yes/NO) ")
if input_user_answer.lower() in ["yes", "y"]:
print("🆗 Keeping current version of save file")
save_file = open(SAVE_FILE, "a")
save_file = open(DEFAULT_PROMPT_RESULT_FILE, "a")
else:
print("🆗 Deleting previous save file")
os.remove(SAVE_FILE)
os.remove(DEFAULT_PROMPT_RESULT_FILE)
print("🔨 Creating new save file")
save_file = open(SAVE_FILE, "a")
save_file = open(DEFAULT_PROMPT_RESULT_FILE, "a")
else:
print("🔨 No previous backup, creating a new save file")
save_file = open(SAVE_FILE, "x")
save_file = open(DEFAULT_PROMPT_RESULT_FILE, "x")

# Ask to user for component details: name, copyright, license, repository hyperlink
print("🆗 Let's get the details of the components to add!")
Expand Down Expand Up @@ -115,19 +111,19 @@ def check_value(value):
continue

# Copyright assigned to the component is optional
input_component_copyright = input("✏️ Copyright of the component ('bye' to exit, '?' if unknown): ")
input_component_copyright = input("✏️ Copyright of the component ('bye' to exit, '{escape}' if unknown): ".format(escape=DEFAULT_AVOID_FIELD_SYMBOL))
if not check_value(input_component_copyright):
continue
check_exit(input_component_copyright)

# Version of the component is optional
input_component_version = input("✏️ Version of the component ('bye' to exit, '?' if unknown): ")
input_component_version = input("✏️ Version of the component ('bye' to exit, '{escape}' if unknown): ".format(escape=DEFAULT_AVOID_FIELD_SYMBOL))
if not check_value(input_component_version):
continue
check_exit(input_component_version)

# Add the new entry in storage file
save_file.write(input_component_name + SAVE_FILE_DELIMITER + input_component_repo + SAVE_FILE_DELIMITER + input_component_license_name + SAVE_FILE_DELIMITER + input_component_copyright + SAVE_FILE_DELIMITER + input_component_version)
save_file.write(input_component_name + DEFAULT_PROMPT_RESULT_FILE_DELIMITER + input_component_repo + DEFAULT_PROMPT_RESULT_FILE_DELIMITER + input_component_license_name + DEFAULT_PROMPT_RESULT_FILE_DELIMITER + input_component_copyright + DEFAULT_PROMPT_RESULT_FILE_DELIMITER + input_component_version)
save_file.write("\n")
components_added += 1

Expand All @@ -146,16 +142,16 @@ def check_value(value):
# Clean up
save_file.close()
print("\n")
print(f'🎉 Operation completed! Find your result file at "{SAVE_FILE}" with {components_added} new component(s)! 🎉')
print(f'🎉 Operation completed! Find your result file at "{DEFAULT_PROMPT_RESULT_FILE}" with {components_added} new component(s)! 🎉')
if components_with_missing_licences > 0:
print("\n")
print(f'❗ But beware you have {components_with_missing_licences} components without managed licenses, you shall fix the result file with suitable names and URL ❗')
print("👉 Please refer to either https://opensource.org/licenses or https://spdx.org/licenses/ 👈")
print("🧡 You can also submit an issue or a pull request to manage new licences: https://github.com/Orange-OpenSource/floss-toolbox/issues/new 🧡")

# Some figures
result_file = open(SAVE_FILE, "r")
reader = csv.reader(result_file, delimiter=SAVE_FILE_DELIMITER)
result_file = open(DEFAULT_PROMPT_RESULT_FILE, "r")
reader = csv.reader(result_file, delimiter=DEFAULT_PROMPT_RESULT_FILE_DELIMITER)
stats = defaultdict(int)

for i, line in enumerate(reader):
Expand Down

0 comments on commit c29a4bf

Please sign in to comment.