events/m4: Add meeting notes

Signed-off-by: Teodor Dutu <[email protected]>

website/docs/events/m4.md

@@ -46,7 +46,310 @@ If you plan to attend, please fill [this form](https://forms.gle/nLmRaEcYTbP7zyi
 
 ## Conclusion
 
-To be filled after the event
+### Open Education Hub: Overview, Goals, Vision, Values, Results so Far - RăzvanD
+
+RD: Today we will show what we've done, and we'll collect feedback from you.
+How will technology affect education?
+We want to make teaching great again, it should be more about mentoring, and less about repetitive tasks.
+
+RD: There's a lot of bureaucracy when teaching that can be automated easily.
+Content development shouldn't waste time each year, we should only improve what we have.
+We want to have everything already there.
+
+RD: We use the OER guides in this project.
+We started with a few people, we gathered some more along the way and now we want to gather even more.
+Iteration makes perfect.
+
+### Methodology for Open Education - RăzvanN
+
+RN: Educational content should be free and available to everyone.
+There are creators and consumers.
+The methodology treats these aspects.
+
+RN: I'll talk about the content creation, and Sergiu will talk about the infrastructure.
+One will go to the Open Education Hub GitHub and will find lectures/content about biology, history, etc.
+We present guidelines on how to create content, how to update it, and how to use it.
+
+RN: We structure content into the following: reading, media, guides, drills, slides, and projects.
+Drills are simple exercises, while projects are broader.
+You need to use more topics and information to solve them.
+
+RN: We then state what content types we need to use for a specific activity.
+The activities are lectures, practical sessions, assignments, self-study, tests.
+The mapping can be one-to-one or one-to-many.
+For example, for lectures, multiple types of content can be included (e.g. slides, media, drills, guides).
+
+RN: Find the repository structure in the template.
+We chose this structure so one can choose what chapters one would teach.
+We keep the entire collection of chapters and you can choose the topics.
+Ideally, a script will extract whatever you want and will publish it to a website.
+
+RN: As for our next steps, we create these tools to do all these stuff automatically, and extend the methodology for other topics.
+
+### Infrastructure for Open Education - Sergiu
+
+SW: Servers go brrrr.
+Setting up classes takes a lot of teachers' time.
+Another issue that we have is integrating feedback.
+
+SW: Our goal is to make the teacher's job easier.
+We want to focus more on the interaction with the students, and less on setting up classes.
+We want to automate everything.
+
+SW: We want to build content that is OER.
+To do this we have to have the content editable by regular users, mostly text format files: slides, tasks, assignments, quizzes.
+The slide you are looking at is actually a markdown file.
+
+SW: The OER builder receives a configuration recipe, and the content, and it servers an HTML content.
+We use HTML because it's text and easy to host.
+
+SW: We use Docusaurus, which converts markdown into HTML easily.
+Reveal-md generates slides from markdown files.
+
+SW: After building the content, we want to distribute it.
+By using HTML and git, we have the advantage that this content can be viewed in the browser.
+Automatic publishing is also part of this flow.
+We can publish content on both GitHub and GitLab.
+
+SW: Others will find the content and they can suggest improvements.
+The suggestions come as pull requests, the owner of the repository is responsible for "accepting" the suggestion, but apart from that, everything is published automatically.
+SW: After building and publishing the content, we need to check the students' assignments.
+They need instant feedback.
+So we built VMChecker.
+It uses containers for easy checking.
+
+SW: We integrated VMChecker with a familiar interface - Moodle.
+Students can upload assignments via GitLab or Moodle, both of them allowing them to receive instant feedback from VMChecker on Moodle.
+A teacher creates an assignment in Moodle and that's it.
+
+### Repository Walkthrough
+
+#### Operating Systems - Teo
+
+TD: We have all the content in the [Operating Systems repo](https://github.com/open-education-hub/operating-systems).
+It's rendered [using Docusaurus](https://open-education-hub.github.io/operating-systems/) as Sergiu has just explained.
+
+TD: Some of the things to note about the content are first that slides are embedded into the website [like so](https://open-education-hub.github.io/operating-systems/Lecture/Software-Stack).
+These slides are stored in Markdown format in the repo on GitHub and then "compiled" and rendered on this website so anyone can browse them.
+
+TD: Another thing to note is related to our assignments.
+We have 5 up to now and we're planning to keep adding assignments so that at some point we may choose a given number of them at the start of the semester and use those for evaluation.
+All assignments are graded via the new VMChecker that Sergiu has also touched on.
+
+TD: Now moving back to the repo, RăzvanN told you that students are encouraged to contribute.
+We award cryptocurrency rewards for this, namely SmileyCoins.
+Initially, we're doing this manually, but for tracking we-re also assigning labels to these PRs.
+Being just a token coin, the amounts we're awarding are still also symbolic: 1 and 5, depending on the PR's complexity.
+
+RD: Keep in mind that the rendered website is produced automatically from the GitHub repo.
+
+TD: yes, when a contribution is made to the GitHub repo, there is an automated system in place that publishes the updated version of the website.
+This way, students always have access to the most up-to-date materials.
+
+#### Security Summer School - Gabi
+
+GM: We migrated the SSS to the OpenEduHub format.
+Now we'll see a "before and after" comparison.
+We started with the [wiki page (2014-2020)](https://security.cs.pub.ro/summer-school/wiki/) where we had to manage access to people that could contribute and there was no way of reviewing changes.
+
+GM: Afterwards we switched to an intermediate format, not so different from the current Open Education format.
+It used [Hugo, a theme for Docsy](https://security-summer-school.github.io/).
+We had some problems with rendering different types of content.
+
+GM: Right now we have SSS on [Open Education Hub](https://open-education-hub.github.io/binary-security/).
+We follow the methodology that RăzvanN talked about, we have multiple chapters with different types of content.
+
+#### Computing and Calculus for Advanced Statistics - Eggert
+
+EKH: I teach a mostly self study course at University of Iceland.
+We had this material, it was tough to maintain, the person responsible for the content is about to retire.
+
+EKH: Our task was to take this content in LaTeX form and create markdown from it.
+We thought about using automatic conversion tools from LaTeX to markdown, but each tool we tried required a lot of configuration.
+
+EKH: Originally, students created parts of this content, each had a different style, different structure, some had images.
+EKH: After lots of trials, we [got it to work](https://open-education-hub.github.io/ccas/).
+
+#### Questions
+
+Marius: Is there a nicer looking content already available?
+
+GM: We have something nicer in the [Operating Systems course](https://open-education-hub.github.io/operating-systems/).
+
+---
+
+Marius: Can you for example modify the background?
+
+RD: Yes.
+We will integrate all sorts of renderers that allow this.
+We didn't have the time yet for this.
+
+---
+
+Cătălin: Do you have the option to change the theme?
+
+RD: Yes, `reveal-md` allows this.
+We propose here more of a methodology, the outcome itself is now not the best it can be, but it works.
+We can make it look prettier.
+
+---
+
+Cătălin: Will the system support raw HTML?
+
+RD: Yes.
+
+---
+
+Dragoș: Let's say I'm a student, the teacher created a content, can I download the content?
+
+RD: Yes, everything is in a repo.
+
+---
+
+Bogdan: What you presented are entire courses.
+Could it work for some educational content (not an entire course).
+
+RD: There is going to be a central repo that contains all of them.
+Afterwards you make a copy/fork and take the course.
+Any sort of teaching content should be here.
+The process should apply anywhere.
+Children, highschool teachers, trainers, anybody.
+Based on the config, you choose what to extract from an entire topic.
+
+---
+
+Costin Carabaș: Do you have support for different languages?
+
+RD: Not currently, we want the content to be internationalizable.
+Now it's in English because it was the most appropriate one to start with.
+Some might complain about this, but in the future, we will make all items internationalizable?
+
+---
+
+Adi Stanciu: How do you cover overlapping content?
+RD: It's not obvious.
+Someone should not create a new class simply because you want things differently there.
+On the other hand, if you need content from multiple classes, you specify this in the config.
+
+Adi Stanciu: How do you help teachers select the proper content for the class?
+Let's say you have content, different levels of detail, and conflicting ideas.
+
+RN: The idea here is that you have the chapters organized on topics.
+You go on the topics, you have more reading files, drills for them and select the ones that are fit for you.
+
+---
+
+Lucian: How much time did it take to migrate the OS content to this format?
+Maybe this could benefit from automation.
+
+RN: It's hard to tell because we didn't have all the content and then we deployed.
+We built the content from zero, so it's different.
+It's not publishing the content that is complicated.
+The most time-consuming part is to think through new ideas and how to structure things, what to add, what to remove, what to avoid, and how to make it better.
+
+---
+
+Radu: How does this format handle team assignments?
+
+RN: Only one can submit.
+
+### Workshops
+
+#### Creating and Deploying an Open Education Hub Repository - Vlad
+
+TODO
+
+#### Automating Assignment Evaluation with VMChecker - Mihai
+
+MB: Before the presentation, I was asked if there are any actual examples of assignments that were used in production.
+I pointed them to the SO repository.
+
+MB: Presentation about vmchecker
+
+MB: Demo for setting up testing infrastructure
+
+AM: Most institutions in Romania use Moodle, so it was a good choice for support with the plugin
+
+AM: Do you integrate with the Moodle repository?
+
+AM: When Moss will be integrated do you plan on comparing against solutions from the previous year?
+
+#### Adding Linters and GitHub Actions in Your Educational Repository - Gabi + Alex
+
+TODO
+
+#### Developing Evaluation Content with Open Education Hub (quizzes) - Ștefan + Teo
+
+SJ: TODO
+
+Vlad Traistă-Popescu: Does this script support images?
+SJ: No, not yet.
+
+Vlad Traistă-Popescu: We did something similar, it uploads the image as `base64`` and Moodle knows how to handle those.
+
+Multiple types of questions (not just multichoice) should be supported.
+
+The workflow is still quite technical (cloning a repository, running a python script, importing stuff into Moodle). The user should not even have to deal with the idea of "question format", it should be a drag-and-drop.
+
+#### Attracting and Engaging Students During Live Activities (Lectures, Seminars) - Liza
+
+Costima Rughiniș: name tags for students and random questions directed at them during the lecture.
+
+---
+
+Cecilia: working in groups, but there's always a group where only one guy is working.
+
+LB: This is good for seminars, but not so much for lectures.
+
+---
+
+CR: Analyzing a dataset based on the answers of the audience.
+
+LB: Analyzing quiz answers and explaining/asking why/what they got wrong
+
+#### Statistics and Metrics for the Teaching and Learning Processes - Alex
+
+After setting up the OpenEducationHub repository, it's essential to consider how to assess its effectiveness.
+The main topics we will discuss will be about:
+
+##### Collecting Metrics and Feedback
+
+At the most fundamental level, the gradebook serves as the primary form of feedback, indicating how well students are performing academically.
+
+In addition to grades, one approach to get insight on how well the subject is going is to implement an end-of-year feedback form that allows students to express their overall satisfaction or agreement with the subject or course.
+This feedback form can be a valuable tool for gathering qualitative insights into students' experiences, opinions, and perceptions of the subject matter and the educational approach.
+
+In addition to the annual feedback forms, setting up weekly feedback forms is essential to promptly capture immediate or "real-time" feedback.
+These forms serve as an effective means to detect moments when students may lose interest or encounter difficulties in comprehending the course materials:
+
+* Brief and Focused: limit the questions to 3-4 to encourage completion.
+* Direct Questions: use closed-ended questions to save time.
+* Optional Expression: forms are a way for students to share thoughts and are not mandatory metrics.
+
+##### Processing Feedback
+
+Raw feedback isn't always ideal because it can be emotionally charged, potentially affecting you personally.
+One approach to address this is to have a third party review and filter out aggressive or non-constructive feedback, leaving behind only valuable suggestions for improvement.
+
+However, this filtering process can result in a loss of information, making it harder for you to fully comprehend certain opinions.
+To address this, consider providing students with the option to include their email addresses in the feedback form.
+This enables you to reach out to them later for clarification and a deeper understanding of their perspective.
+
+In essence, this approach closes the feedback loop and encourages engaged students to contribute more effectively by facilitating a two-way conversation for better communication and comprehension of their feedback.
+
+##### Using Metrics
+
+Analyzing correlations between feedback and grades can provide valuable insights into necessary changes:
+When students who have received lower grades in specific areas consistently provide similar feedback, it signals potential deficiencies in the educational materials for those areas.
+When students with high grades also provide positive feedback, it suggests that increasing engagement for other students may be beneficial.
+Conversely, if most students express positive assessments of the subject but exhibit divergent grades, it raises the possibility that the provided materials may not be sufficient for a complete understanding of the concepts.
+
+Feedback itself can be used as a transparency tool.
+This involves sharing with students the ideas and suggestions provided by previous student cohorts and engaging in discussions about the improvements that have been made in response to this feedback.
+
+##### Using the Open Education Hub Methodology and Infrastructure Beyond Computer Science Topics - RăzvanN
+
+TODO
 
 ### Takeaways