GSoC 2022 Final Submission: RTEMS Release Generator

This post presents the final deliverables of my GSoC project for summer 2022 with the RTEMS Project organization. Throughout the summer, I have been working on a new release notes generator for the RTEMS release workflow.

This project started in GSoC 2018 by Danxue Huang and it was mainly concerned with building a program to interact with the RTEMS Trac API to fetch tickets for a particular release as well as organize the release in the JSON format and then simply convert it to HTML tables. This year, I picked up the project and worked on generating native HTML and reStructuredText from the fetched data as well as the performance of the generator.

In the first part of GSoC, I was mainly concerned with fixing semantic and style bugs in the existing generator, and we looked into what makes a good release notes document in terms of content. By the end of the first half, functional Markdown and HTML generation were working. After that, performance analysis work has been done to increase the overall time of release notes time substantially by parallelizing sending requests to Trac as well as parallel generation of tickets in the target Markup language. Also, work has been done to support the generation of reStructuredText-based PDF release notes files.

One key tradeoff in the design of the generator was which Markup language to use for PDF generation. The two main attractive ones were Markdown and reStructuredText. Each of them had advantages, but neither of them was perfect. On one hand, Markdown generation automation is much easier than reStructuredText, but PDF generation has to be done by first converting into HTML and then using wkhtmltopdf, which produced generation is far from perfect, especially when content is to be split across multiple pages. On the other hand, reStructuredText generation has many hurdles in automating its generation, but the PDF conversion tools are readily available and have much better and generally appealing documents. Both are supported in the current generator with Markdown being the default.

Trac has a Markup language that we'd like to transform into the Markup language we use. However, one of the most challenging hurdles in implementing the generator was dealing with tickets that had Trac Markup syntax errors. One straightforward solution is to figure out which tickets have problems and fix them manually on fetching. We use regular expressions to replace some of the common manually-found syntax errors, which turns out to work quite well. Surely, some novel syntax errors will be made in the future, and they may break parts of the generated documents, in which case it may be essential to fix them. Another solution is to treat all the fetched data as plain text and don't consider the Trac Markup language whatsoever, but that would prevent us from rendering code and other snippets in the document, which is not desired.

The code for the new generator can be found in the same repository used throughout GSoC, here. It's a fork that uses the code from GSoC 2018 as a base and builds up on it.

I plan to keep in touch with the RTEMS community for any bugs to fix with the project as well as add any desired features. I am grateful to meet my mentors and other RTEMS community members who have shown nothing but patience and helpfulness throughout my participation in GSoC.

The following are other blog posts written throughout the participation.

• Introduction, scope, and objectives

• Design & performance investigation of the RTEMS release notes generator

• Midterm deliverables & checkpoint