GSoC Mid Term Recap

Aazam ThakurAazam Thakur
8 min read

Table of contents

Overview

This article details the work done by Aazam Thakur (a recent engineering graduate in Artificial Intelligence and Data Science) as part of the Google Summer of Code 2025 during the first half of the term, under the openSUSE organization for the project titled “Logdetective for openSUSE/OBS” under the mentorship of Daniel Garcia.

The goal of the project is to research implementations on utilizing the LogDetective model for openSUSE/obs workflows. Logdetective is an AI modal developed by fedora fine tuned for build log failures. OBS (OpenSUSE Build Service) is a package distribution platform managed by openSUSE. The aim is to analyze the failed build logs on OBS using the log detective modal. All work related to the project can be found in this repository

Deliverables

During the first half the following tasks were completed:

  1. Creating a python script to fetch build logs and run logdetective model

  2. Documenting installation and README.md

  3. Creating an osc plugin to integrate logdetective model as a command

The task for the first month was to understand the codebase of the LogDetective project and create a python script that would utilize the LogDetective API. Running the get-logs.py file:

(venv) 3c11c8c06131: python3 get-logs.py openSUSE:Factory -p blender -e -l
🔍 Checking failed builds for package 'blender' in project 'openSUSE:Factory' (openSUSE:Factory/blender)...

Running 'osc results -f openSUSE:Factory/blender'...
The failed builds are:
standard             x86_64     blender                        failed
📥 Downloading: https://build.opensuse.org/public/build/openSUSE:Factory/standard/x86_64/blender/_log
✅ Saved to logs/openSUSE:Factory/blender_standard_x86_64.log
Running local log-detective on logs/openSUSE:Factory/blender_standard_x86_64.log...
🕵️ Analysis saved to explain/openSUSE:Factory/blender_standard_x86_64.txt

We can see from the output, we create a subdirectory as per the project name and save the corresponding failed build logs that we fetch from OBS. Similarly a subdirectory is created in the explain folder for the project.
The -p flag specifies a single package (without it, we fetch all failed packages in the project)
The -e flag specifies the logdetective model to analyze the downloaded log file.

The -l flag specifies the script to use a local logdetective model instead of calling the logdetective API. Snapshot of the explained failed build log by logdetective.

Explanation:
Snippet No. 0: This snippet shows the compiler command being executed to build the package 'blender'. It includes various flags, include directories, and libraries required for the build.

Snippet No. 1: This snippet indicates that the build process has failed due to a subcommand failure.

Snippet No. 2: This snippet further explains that the build process encountered an error with a bad exit status.

Snippet No. 4: This snippet is a repetition of the error message indicating the RPM build failure.

Snippet No. 5: This snippet indicates the start of VM interaction for handling the build failure.

Snippet No. 6: This snippet shows the command to power off the virtual machine. It is an indication that the build process is being terminated.

The build process of the 'blender' package has failed due to a subcommand failure during the compilation step. The exact nature of the failure is not indicated by the given log snippets. A power off command has been issued to terminate the virtual machine during the build process.

To diagnose the failure in more detail, it would be necessary to examine the complete build log or the specific error messages generated by the compiler. Potential solutions could include modifying build dependencies, updating compiler versions, or fixing build errors in the package source code. Without further information, it is impossible to recommend a specific solution.

Recommendation:
To diagnose the build failure in more detail, the complete build log or the specific error messages generated by the compiler should be examined. Potential solutions include modifying build dependencies, updating compiler versions, or fixing build errors in the package source code. If the issue cannot be resolved, it might be necessary to consult the upstream package maintainers or seek assistance from the open-source community.
Response certainty: 91.93%

The next significant task was the development of an osc plugin to incorporate the features of the previously created script and integrating it with osc itself to streamline workflows. The osc-log.py file serves as the basis for the plugin. This served as a complex task in order to utilize it as an internal process instead of calling osc each time and initializing it.

This will be run as follows as an example:

osc ld --project openSUSE:Factory --package blender

This outputs a similar response as the script from before. There is an alternative flag to use —local-log, that utilizes the last local build log.

osc ld --local-log

which gives us an output as shown:

The first half gives us the build log, then we receive the analyzed text after passing the build log into logdetective.

Found local build log: /var/tmp/build-root/standard-x86_64/.build.log

--- Displaying local build log from: /var/tmp/build-root/standard-x86_64/.build.log ---
[    0s] Memory limit set to 11615796KB
[    0s] Using BUILD_ROOT=/var/tmp/build-root/standard-x86_64
[    0s] Using BUILD_ARCH=x86_64:i686:i586:i486:i386
[    0s] 
[    0s] 
[    0s] 3c11c8c06131 started "build blender.spec" at Tue Jul  8 10:47:11 UTC 2025.
[    0s] 
[    0s] 
[    0s] processing recipe /openSUSE:Factory/blender/blender.spec ...
[    0s] running changelog2spec --target rpm --file /openSUSE:Factory/blender/blender.spec
[    0s] init_buildsystem --configdir /usr/lib/build/configs --cachedir /var/cache/build --clean --rpmlist /tmp/rpmlist.7k_jxi9r /openSUSE:Factory/blender/blender.spec ...
[    1s] Warning: cross compile not possible due to missing static binaries. please install build-initvm package for that purpose.
[    1s]          check that the right architecture is available for your build host, you need  for this one.
[    1s] unpacking preinstall image openSUSE:Factory/standard/preinstallimage-base [a79f39f76b73f133c32769f6b5440238]
[    1s] tar: run/systemd.pre-switch-root/inaccessible/chr: Cannot mknod: Operation not permitted
[    1s] tar: Exiting with failure status due to previous errors
[    1s] ERROR: unpack failed.
[    1s] ERROR: This is a .zst compressed preinstallimage and tar --exclude .build --exclude .init_b_cache -x failed to unpack.
[    1s] Try to enable zst in tar --exclude .build --exclude .init_b_cache -x or disable preinstallimage.
[    1s] 
[    1s] 3c11c8c06131 failed "build blender.spec" at Tue Jul  8 10:47:12 UTC 2025.
[    1s] 

--- End of local build log ---

🚀 Feeding local build log to logdetective: /var/tmp/build-root/standard-x86_64/.build.log
INFO:logdetective:Loading model from fedora-copr/Mistral-7B-Instruct-v0.2-GGUF
INFO:logdetective:Getting summary
INFO:logdetective:Compression ratio: 0.34782608695652173
INFO:logdetective:Analyzing the text
INFO:logdetective:Log summary: 

            Snippet No. 0 at line #14:

            [    1s] unpacking preinstall image openSUSE:Factory/standard/preinstallimage-base [a79f39f76b73f133c32769f6b5440238]

            ================

            Snippet No. 1 at line #15:

            [    1s] tar: run/systemd.pre-switch-root/inaccessible/chr: Cannot mknod: Operation not permitted

            ================

            Snippet No. 2 at line #16:

            [    1s] tar: Exiting with failure status due to previous errors

            ================

            Snippet No. 3 at line #17:

            [    1s] ERROR: unpack failed.

            ================

            Snippet No. 4 at line #18:

            [    1s] ERROR: This is a .zst compressed preinstallimage and tar --exclude .build --exclude .init_b_cache -x failed to unpack.

            ================

            Snippet No. 5 at line #19:

            [    1s] Try to enable zst in tar --exclude .build --exclude .init_b_cache -x or disable preinstallimage.

            ================

            Snippet No. 6 at line #20:

            [    1s] 

            ================

            Snippet No. 7 at line #21:

            [    1s] 3c11c8c06131 failed "build blender.spec" at Tue Jul  8 10:47:12 UTC 2025.

            ================

Explanation:
1. The package 'preinstallimage-base' failed to unpack during the build process.
2. The error was due to insufficient permissions to create a character device file 'chr' during the unpacking process.
3. As a result, the tar command exited with a failure status.
4. The preinstall image was in .zst format which required enabling zst in the tar command or disabling the preinstall image to resolve the issue.

Recommended solution:
To resolve the issue, you can either enable zst in the tar command or disable the preinstallimage-base package and rebuild it without it.

Enabling zst in tar command:
1. Update your system's tar package to support zstd:

udo zypper addrepo -f https://download.opensuse.org/repositories/devel:/lts150/openSUSE_15.0/devel:/lts150/ openSUSE15.0 sudo zypper install -y tar-zstd
2. Modify the build script to include the zstd flag:
%global _topdir %{_srcdir} %global _builddir %{_topdir}/build %global _buildroot %{_builddir}/%{NAME}-%{VERSION}-%{BUILDID}-%{PLATFORM} %global _rpm_buildroot %{_buildroot}/RPMS %global _rpm_srcdir %{_topdir}

%prep ...

%build make %{?_smp_mflags} -C %{_builddir} %{?_srcdir}/CMakeLists.txt tar -xf %{_source_file} --no-same-permissions --compress-level=9 -C %{_builddir} --xz-compress --zstd -C %{_builddir} --strip-components=1 -o %{_buildroot}/%{NAME}-%{VERSION}-%{BUILDID}-%{PLATFORM}.rpm.tar.xz

%install make install -C %{_builddir} DESTDIR=%{_buildroot}/%{NAME}-%{VERSION}-%{BUILDID}-%{PLATFORM}

%post ...

%files ...
Disabling preinstallimage-base:
1. Remove the dependency on preinstallimage-base from the package spec file:

%global _topdir %{_srcdir} %global _builddir %{_topdir}/build %global _buildroot %{_builddir}/%{NAME}-%{VERSION}-%{BUILDID}-%{PLATFORM} %global _rpm_buildroot %{_buildroot}/RPMS %global _rpm_srcdir %{_topdir}

%prep ...

%build make %{?_smp_mflags} -C %{_builddir} %{?_srcdir}/CMakeLists.txt

%install make install -C %{_builddir} DESTDIR=%{_buildroot}/%{NAME}-%{VERSION}-%{BUILDID}-%{PLATFORM}

%files ...

%post ...

%preun ...

%postun ...

%changelog ...

%description ...

%name %{NAME} %version %{VERSION} %description %{_descr}

%package rpm-build %description rpm-build package %files /dev/null


After making the necessary changes, rebuild the package to resolve the issue.
Response certainty: 98.52%

Observations

During the first half of the term, after trial and error involving testing and iterations, we noted the following:

  • It is best to download and use the logdetective model locally instead of calling the API, due to much faster inference (API takes several minutes)

  • Currently log detective utilizes the “Mistral-7B-Instruct-v0.2-GGUF”. We have found this model to perform poorly, this is expected as the model is a Dec 2023 release. The minimum RAM requirement to run this model without swap is 8GB.

  • Deploying logdetective on a private instance on openSUSE (tumbleweed here) currently requires more investigation as it heavily requires the use of NVIDIA GPUs.

Future Scope

For the following weeks, work will be done to further investigate techniques and research about different models to replace the current mistral model. Benchmark testing had been conducted by logdetective devs where they compared results and currently found the gemini model to perform with the highest accuracy and usefulness of output.

Another task is to further research into incorporating it into CI/CD processes such as GitHub actions. Once resources have been potentially acquired for deployment, there is a possibility to run a logdetective instance for openSUSE itself and fine tune it further.

Conclusion

While the current development and utilization of this model is in the earlier stages. With the rapid pace of development in AI models, there is a huge opportunity for this project to cut down development time and improve productivity.

0
Subscribe to my newsletter

Read articles from Aazam Thakur directly inside your inbox. Subscribe to the newsletter, and don't miss out.

gsocPythonOpen SourceAI

Written by

Aazam Thakur
Aazam Thakur

I'm a software engineer from the city of Mumbai. Currently in my majoring in Artificial Intelligence and Data Science. I am an OSS contributor and love participating in hackathons.