Working with Backstage Software Templates

Recently I have been working with https://backstage.io/ , specifically the Backstage Software Templates feature

I noticed a few things and learned a lot of things too

Here are some short notes 🗒️ 📝 on what I noticed and also what I learned

• Great Developer Experience (DevEx) when building Backstage Software Templates

  • I could use the Template Editor provided by Backstage. I used the Load Template Directory feature to load my template and also be able to reload the template. I was also able to execute the template multiple times and see the results - any files, any logs etc 😁😄😃😀. I was doing all the development in my local - running Backstage in local, running Visual Studio Code (VSCode) in local, with the directory containing the Backstage Software Template in my local, and using Visual Studio Code (VSCode) to edit the Backstage Software Template though one could edit on the browser itself. It's kind of like an IDE of sorts :)

• Error handling is done really well in Backstage :)

  • When developing Backstage Software Templates, whenever I made mistakes, I was able to notice error messages that helped me clarify and understand what the error was. If there was a user side error, there was a 4xx HTTP response, and if it was server side error, there was a 5xx HTTP response along with error message in the response I think, and hence on the front end. For 5xx HTTP responses, I was able to see the issue and error in the terminal where my Backstage backend was running. The backend was logging all the HTTP requests it was getting and any errors that occurred. Errors were also shown at the bottom of the Backstage Software Template that I was executing from Load Template Directory Development feature. Again, like I said, it's kind of like an IDE of sorts :D :)

• I noticed that it was hard to do the testing

  • I had to do a lot of testing manually. There are a few high level ideas 💡 that I'm thinking of, to automate the testing. It's not completely solutionized or implemented. I have to research and check the feasibility of the solutions. But I'll put it up here, at the bottom

• I noticed that it was hard to define constants or variables of sorts that I can reuse

  • Backstage Software Templates use Nunjucks templating and allow you to use Nunjucks and their own custom stuff, say filters, functions etc. All are documented 📃📄📑 in https://backstage.io/docs/features/software-templates/writing-templates and Nunjucks stuff is documented at https://mozilla.github.io/nunjucks/ and https://mozilla.github.io/nunjucks/templating.html . For constants, I think we just need variables and for that, we can use some plugin, for example, if the variable or constant is ONLY required during processing of the form input in the sequence of steps in the Backstage Software Template, then you can use a custom action, a custom plugin, like this - https://www.npmjs.com/package/@gabtec/backstage-action-var-set , which helps you set variables. Not sure if the variable values support all kinds of values, like number, string, object, array etc

• I didn't disable the authentication and authorization feature and this was making development harder

  • I didn't know how to do this. I'm yet to figure this out. I'm stubborn in not asking my colleagues who probably already know something or figured it out already and have implemented it too for their local development. But it's not part of any "Contributing Guide"

  • In our custom production Backstage instance, we have of course enabled Authentication and Authorization and we use Keycloak for it and Single Sign On enabled in Keycloak 🔑🔐. The thing is, somewhere there's some timeout / session expiry / token expiry which causes the session to get expired and it then causes errors showing up in the front end - saying that Auth failed. This is pretty annoying. Ideally, for local development, unless you are working on Authentication and Authorization, Auth can be disabled, so that it's not bugging you and hindering you. I have had to refresh my page and login multiple times. The main issue is not even refreshing and logging in multiple times, the main issue is, I have a big form, with multiple stages and each stage getting a lot of data - so, once this auth error happens, I just lose all this data on refresh. And no, if I use a separate tab to login and retry stuff without refresh, it doesn't work, I don't know why. For each new 🆕 tab, I need to login. For some reason the login session / token is not stored I think 🤔💭, say, in the browser storage, for it to work across browser tabs. And no, I'm not using something like Firefox containers (or any Chrome stuff) with separate containers and different container tabs to open the Backstage UI causing it to be oblivious about the other tabs / be isolated from other tabs and the storage of other containers and their tabs. Due to this auth error, I end up having to type in and key in all the data again to the form and keep clicking "Next" and then click "Create" to run the Software Template's steps

• I noticed there was also a LOT of code repetition in the Software Template. Some ideas I thought 💭 of for fixing this are

  • Try to use YAML language features, like Anchors and Alias

  • Try to codify things

    • Use a templating language to generate the Backstage Software Template

    • Use programming language to generate the Backstage Software Template.

      • So, kind of like using a transpiler or some processor of sorts to process some easy to understand thing to the complex Backstage Software Template which has lot of repetition. Since it's hard to work with repetitive code, you can work with non-repetitive code which is easy to understand and modify without any errors

      • With programming languages, we can write tests too 👌 ;) And do configuration testing

      • Problems / Drawbacks

        • Harder and Complex, but sounds pretty awesome 😍🤩🙌 and powerful 💪 to me. It might make life just easier, or just harder or very much harder, lol

        • We need tools ⚒️🧰🛠️ for this or else it would still be hard to use whatever new tool/system 🪚🔨 we use. If it's a existing programming language or some existing configuration language like YAML etc, then fine. If we create some Domain Specific Language, gotta be careful about it!

• It was easier to make UI changes in the Backstage Software Template form once it became clear that the forms use react-jsonschema-form library for the form creation and working. The official website for that project is https://rjsf-team.github.io/react-jsonschema-form/ and it has a playground and there are docs at https://rjsf-team.github.io/react-jsonschema-form/docs/ . But you may not be able to use all the feature in https://rjsf-team.github.io/react-jsonschema-form/ . Why? I don't know, gotta check. For example, you can't use definitions from what I understand till now.

What I'm yet to try/do etc

• Validation of Form fields

  • I'm gonna check react-jsonschema-form docs and Backstage Software Template docs and then see how to do validation like regular expression based validation for string inputs and then almost any kind of custom validation. I might have to write some custom code as a plugin and do it. Or even a custom input field. Let's see :)

Solutions to Testing Backstage Software Templates easily

• Consider checking Configuration Testing Tools and see if they can be used here for any testing

  • For example https://www.conftest.dev/

• For an End to End Test, we need to run a Backstage instance, and then use the Backstage Software Template and execute it after giving it all the inputs. For this, we can use multiple automation. One automation to just simply run the Backstage instance. Another automation to mimic user behaviour, a user who is using a browser and interacting with the Backstage Software Template. For this mimicing behaviour, we can use Browser automation tools 🧰🛠️⚒️, libraries, frameworks, systems and services. For example, Selenium, Appium etc

• For an easier and faster test, especially for the local and also for the CI, and also to have more test cases without losing out on time when using End to End test, we can look for unit test and integration test kind of stuff

  • For unit test, we need to look at the unit we are trying to test. The subject under test (SUT) basically. For example, one unit to test is - the list of steps to run once the Backstage Software Template form is completed by the user ☑️✅✔️. For testing the steps or processes that happen once the form is submitted, we can look at what actions are used in the steps. If the steps are fetch:template etc, it should be easy, but others should also be easy. So, basically, we are not trying to test the step itself, we are just trying to ensure that the output of the step is correct and what we want. So, basically, the subject under test is the template that you write ✍️🖊️✏️✒️🖋️ which is a templatized thing. DO NOT end up testing the steps - steps need to tested by the creator of the actions in the action source code etc. If that's you, then since you are creating the action, you need to test it. Now, so, what do you do to test the template? Take the template and run it through the sequence of steps that you have defined and see if it does exactly what you expected it to do. For example, if you wanted it to create files after templatizing, then have an expected set of files and compare the actual output of files to the expected. Now, how do you run the sequence of steps? Well, we need the actual Backstage runner to run these steps and also need to ensure that all the required actions are available and installed. One thing we can do is - instead of running the whole Backend server of Backstage, you could just create a tool to get the parts of Backstage that does this - running of the workflow of steps. I don't think such a tool already exists, so, you need to create one. I plan to create one. If I do, I'll share it in my blog :) :D 😁😄😀😃 You could import the Backstage repo libraries etc as it's all open source and then use it to do what Backstage backend itself does :) This way, you can just use a CLI like tool, which is lightweight and faster and easier to use and test your code 🧑‍💻👩‍💻👨‍💻 instead of using slow browser based testing

  • Based on the above idea 💡, you can extrapolate it and try to do the same for integration testing