[GH-ISSUE #6033] Zip the supplied profiles #3161

Closed
opened 2026-05-05 09:47:12 -06:00 by gitea-mirror · 12 comments
Owner

Originally created by @birdie-github on GitHub (Oct 4, 2023).
Original GitHub issue: https://github.com/netblue30/firejail/issues/6033

The number of featured profiles has grown out of hands, there's now over 1200 of them, which results in a heavy filesystem fragmentation.

Please consider putting all of them in a single archive of your choice, zip would be fine but it's up to you.

Originally created by @birdie-github on GitHub (Oct 4, 2023). Original GitHub issue: https://github.com/netblue30/firejail/issues/6033 The number of featured profiles has grown out of hands, there's now over 1200 of them, which results in a heavy filesystem fragmentation. Please consider putting all of them in a single archive of your choice, zip would be fine but it's up to you.
gitea-mirror 2026-05-05 09:47:12 -06:00
Author
Owner

@ghost commented on GitHub (Oct 4, 2023):

Due to Firejail's design that would need code changes, potentially adding dependencies.

Can you elaborate on what you describe as heavy filesystem fragmentation? Did you use something like e4defrag to determine fragmentation scores on ext4 or similar?

<!-- gh-comment-id:1747724968 --> @ghost commented on GitHub (Oct 4, 2023): Due to Firejail's design that would need code changes, potentially adding dependencies. Can you elaborate on what you describe as `heavy filesystem fragmentation`? Did you use something like `e4defrag` to determine fragmentation scores on ext4 or similar?
Author
Owner

@kmk3 commented on GitHub (Oct 4, 2023):

Currently each profile is an individual plain text file, which makes them easy
to find, inspect, compare and manipulate in general using standard unix
utilities. For example, comparing foo.profile to foo.local in any text editor.

Compressing all profiles into one file would reduce accessibility and make
those actions more complicated than they need to be.

Compressing each individual profile would probably not make a significant
difference in space used (given how small most profiles are) and would probably
still result in the same amount of fragmentation.

Related:

<!-- gh-comment-id:1747751201 --> @kmk3 commented on GitHub (Oct 4, 2023): Currently each profile is an individual plain text file, which makes them easy to find, inspect, compare and manipulate in general using standard unix utilities. For example, comparing foo.profile to foo.local in any text editor. Compressing all profiles into one file would reduce accessibility and make those actions more complicated than they need to be. Compressing each individual profile would probably not make a significant difference in space used (given how small most profiles are) and would probably still result in the same amount of fragmentation. Related: * #4871
Author
Owner

@birdie-github commented on GitHub (Oct 5, 2023):

Can you elaborate on what you describe as heavy filesystem fragmentation? Did you use something like e4defrag to determine fragmentation scores on ext4 or similar?

A ton of small files prevent larger file from occupying continuous blocks of storage.

Currently each profile is an individual plain text file, which makes them easy
to find, inspect, compare and manipulate in general using standard unix
utilities.

A zip file with profiles does not change that.

For example, comparing foo.profile to foo.local in any text editor.

unzip exists.

Compressing each individual profile would probably not make a significant
difference in space used (given how small most profiles are) and would probably
still result in the same amount of fragmentation.

I did not talk or mean compressing them individually, I want to see a single zip file with all the profiles within.

<!-- gh-comment-id:1748954303 --> @birdie-github commented on GitHub (Oct 5, 2023): > Can you elaborate on what you describe as heavy filesystem fragmentation? Did you use something like e4defrag to determine fragmentation scores on ext4 or similar? A ton of small files prevent larger file from occupying continuous blocks of storage. > Currently each profile is an individual plain text file, which makes them easy to find, inspect, compare and manipulate in general using standard unix utilities. A zip file with profiles does not change that. > For example, comparing foo.profile to foo.local in any text editor. unzip exists. > Compressing each individual profile would probably not make a significant difference in space used (given how small most profiles are) and would probably still result in the same amount of fragmentation. I did **not** talk or mean compressing them individually, I want to see a single zip file with all the profiles within.
Author
Owner

@ghost commented on GitHub (Oct 5, 2023):

From neutral ground there's probably a way around some of the practical issues raised here. I'd imagine it would be quite feasible to use gzip tools like zcat, zdiff, zgrep, zegrep etcetera for administrative tasks.

I did not talk or mean compressing them individually, I want to see a single zip file with all the profiles within.

That states your feature request crystal clear, thanks for elaborating.

A ton of small files prevent larger file from occupying continuous blocks of storage.

Although in theory this is a valid assumption, having actual facts on the numbers involved would help putting things in context. I ran a quick e4defrag fragmentation score for /etc/firejail on my Arch Linux box (using Firejail from git):

$ sudo nice -n 19 e4defrag -c /etc/firejail/ > fragscore.txt
$ cat fragscore.txt
e4defrag 1.47.0 (5-Feb-2023)
<Fragmented files>                                        now/best       size/ext
1. /etc/firejail/discord-common.profile          1/1                 4 KB
2. /etc/firejail/google-earth-pro.profile         1/1                 4 KB
3. /etc/firejail/glaxium.profile                          1/1                 4 KB
4. /etc/firejail/icecat.profile                              1/1                 4 KB
5. /etc/firejail/macrofusion.profile                  1/1                4 KB

 Total/best extents				1280/1280
 Average size per extent			  4 KB
 Fragmentation score				    0
 [0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag]
 This directory (/etc/firejail/) does not need defragmentation.
 Done.

Realizing this is a crude and one-time measurement, things don't seem to look that bad.

Are you seeing high(er) fragmentation scores?

<!-- gh-comment-id:1749247626 --> @ghost commented on GitHub (Oct 5, 2023): From neutral ground there's probably a way around some of the practical issues raised here. I'd imagine it would be quite feasible to use [gzip](https://www.gnu.org/software/gzip/) tools like `zcat`, `zdiff`, `zgrep`, `zegrep` etcetera for administrative tasks. > I did not talk or mean compressing them individually, I want to see a single zip file with all the profiles within. That states your feature request crystal clear, thanks for elaborating. > A ton of small files prevent larger file from occupying continuous blocks of storage. Although _in theory_ this is a valid assumption, having _actual facts_ on the numbers involved would help putting things in context. I ran a quick e4defrag fragmentation score for /etc/firejail on my Arch Linux box (using Firejail from git): ```sh $ sudo nice -n 19 e4defrag -c /etc/firejail/ > fragscore.txt $ cat fragscore.txt e4defrag 1.47.0 (5-Feb-2023) <Fragmented files> now/best size/ext 1. /etc/firejail/discord-common.profile 1/1 4 KB 2. /etc/firejail/google-earth-pro.profile 1/1 4 KB 3. /etc/firejail/glaxium.profile 1/1 4 KB 4. /etc/firejail/icecat.profile 1/1 4 KB 5. /etc/firejail/macrofusion.profile 1/1 4 KB Total/best extents 1280/1280 Average size per extent 4 KB Fragmentation score 0 [0-30 no problem: 31-55 a little bit fragmented: 56- needs defrag] This directory (/etc/firejail/) does not need defragmentation. Done. ``` Realizing this is a crude and one-time measurement, things don't seem to look that bad. Are you seeing high(er) fragmentation scores?
Author
Owner

@birdie-github commented on GitHub (Oct 5, 2023):

Realizing this is a crude and one-time measurement, things don't seem to look that bad.

Let me quote myself again:

A ton of small files prevent larger file from occupying continuous blocks of storage.

Small files punch random holes in your partition. That means larger files have a lot fewer continuous blocks of free space. e4defrag does not show free space fragmentation.

Again, I wasn't talking about own firejail files, I was talking about the free space fragmentation caused by having > 1200 small files all over your partition.

Secondly a directory with so many files can become fragmented itself.

There's a special flag for e2fsck which attempts to fix it -D.

Anyways, I've solved this issue for myself using fuse-zip. Zipped the whole directory, mounted it in /etc/firejail. I'm happy.

<!-- gh-comment-id:1749264370 --> @birdie-github commented on GitHub (Oct 5, 2023): > Realizing this is a crude and one-time measurement, things don't seem to look that bad. Let me quote myself again: > A ton of small files prevent larger file from occupying continuous blocks of storage. Small files punch random holes in your partition. That means larger files have a lot fewer continuous blocks of free space. `e4defrag` does not show free space fragmentation. Again, I wasn't talking about own firejail files, I was talking about the free space fragmentation caused by having > 1200 small files all over your partition. Secondly a directory with so many files can become fragmented itself. There's a special flag for `e2fsck` which attempts to fix it `-D`. Anyways, I've solved this issue for myself using `fuse-zip`. Zipped the whole directory, mounted it in `/etc/firejail`. I'm happy.
Author
Owner

@birdie-github commented on GitHub (Oct 5, 2023):

Lastly, it's highly recommended all over the place, by Unix and Windows filesystem gurus, not to have a large number of files in a single directory:

https://news.ycombinator.com/item?id=19763378

It has a lot of negative implications and the number of firejail files will continue to grow. I don't think you'll ever prune anything since "someone may use it or need it". It almost feels like you could use some sort of database for these files instead, e.g. SQLite but again that's extra dependencies you're not a fan of.

<!-- gh-comment-id:1749271498 --> @birdie-github commented on GitHub (Oct 5, 2023): Lastly, it's _highly_ recommended all over the place, by Unix and Windows filesystem gurus, not to have a large number of files in a single directory: https://news.ycombinator.com/item?id=19763378 It has a lot of negative implications and the number of firejail files will continue to grow. I don't think you'll ever prune anything since "someone may use it or need it". It almost feels like you could use some sort of database for these files instead, e.g. SQLite but again that's extra dependencies you're not a fan of.
Author
Owner

@ghost commented on GitHub (Oct 5, 2023):

@birdie-github

Sure it's recommended not to put large numbers of files in a single dir. Nobody disputed that. But there's a difference between 1200 and 'millions' of files (as the linked HN thread deals with). Personally I can see value in your proposal, which is why I tagged it. Dicussing features is bound to raise arguments for both sides of any fence, no? By closing here, the opportunity to take some time for other people to chime in, play with the idea and have a proper discussion is lost. Feel free to re-open.

<!-- gh-comment-id:1749366136 --> @ghost commented on GitHub (Oct 5, 2023): @birdie-github Sure it's recommended not to put large numbers of files in a single dir. Nobody disputed that. But there's a difference between 1200 and 'millions' of files (as the linked HN thread deals with). Personally I can see value in your proposal, which is why I tagged it. Dicussing features is bound to raise arguments for both sides of any fence, no? By closing here, the opportunity to take some time for other people to chime in, play with the idea and have a proper discussion is lost. Feel free to re-open.
Author
Owner

@birdie-github commented on GitHub (Oct 5, 2023):

Sure it's recommended not to put large numbers of files in a single dir.

That was a weak argument, you're correct. I just personally feel that if an application uses more than a few dozen files in a single directory, something is off unless we are talking about an Indie game where the developer didn't bother to come up with some storage solution for great many files. The vast majority of triple A games use some storage format to pack thousands of files to speed up installation and ease file management.

Discussing features is bound to raise arguments for both sides of any fence, no?

The issue is solely with me. I'm a perfectionist and I love things to be right. However firejail is an open source project developed in your spare time, most likely you're not get paid for it, so, if you, as a developer, oppose it, I think to myself, "You're asking people too much, just be thankful and leave them alone" and retire. It doesn't help that I've been feeling completely dejected and rejected over the past few years, so I question every word I utter.

By closing here, the opportunity to take some time for other people to chime in, play with the idea and have a proper discussion is lost. Feel free to re-open.

It's really up to you. I cannot be the only person seeing the issue and advocating for this solution (it might be completely different), so if there are developers sharing this concern, I'll be happy to be the first to raise it.

I'll reopen it now but again, it's far from something which is pressing or worth addressing right away.

<!-- gh-comment-id:1749520434 --> @birdie-github commented on GitHub (Oct 5, 2023): > Sure it's recommended not to put large numbers of files in a single dir. That was a weak argument, you're correct. I just personally feel that if an application uses more than a few dozen files in a single directory, something is off unless we are talking about an Indie game where the developer didn't bother to come up with some storage solution for great many files. The vast majority of triple A games use some storage format to pack thousands of files to speed up installation and ease file management. > Discussing features is bound to raise arguments for both sides of any fence, no? The issue is solely with me. I'm a perfectionist and I love things to be right. However `firejail` is an open source project developed in your spare time, most likely you're not get paid for it, so, if you, as a developer, oppose it, I think to myself, "You're asking people too much, just be thankful and leave them alone" and retire. It doesn't help that I've been feeling completely dejected and rejected over the past few years, so I question every word I utter. > By closing here, the opportunity to take some time for other people to chime in, play with the idea and have a proper discussion is lost. Feel free to re-open. It's really up to you. I cannot be the only person seeing the issue and advocating for this solution (it might be completely different), so if there are developers sharing this concern, I'll be happy to be the first to raise it. I'll reopen it now but again, it's far from something which is pressing or worth addressing right away.
Author
Owner

@kmk3 commented on GitHub (Oct 22, 2023):

@birdie-github on Oct 5:

Currently each profile is an individual plain text file, which makes them easy
to find, inspect, compare and manipulate in general using standard unix
utilities.

A zip file with profiles does not change that.

For example, comparing foo.profile to foo.local in any text editor.

unzip exists.

Currently I can go into /etc/firejail, open any profile in any text editor and
quickly comment/uncomment lines while debugging a profile. I can do this
multiple times in a row, at any time and without effort and be sure that the
changes will be persistent. When I ask a user (who might not be very familiar
with the CLI) to edit a profile in /etc/firejail for debugging, they are free
to do so in their text editor of choice and they usually manage to accomplish
it without issues.

How would that work with unzip?

I don't see how this would be possible without a significant impact to
usability and accessibility.

@birdie-github on Oct 5:

Again, I wasn't talking about own firejail files, I was talking about the
free space fragmentation caused by having > 1200 small files all over your
partition.

Secondly a directory with so many files can become fragmented itself.

Since /etc is usually mostly consisted of small files, one option would be to
use a small separate filesystem for it with smaller block sizes.

Anyways, I've solved this issue for myself using fuse-zip. Zipped the whole
directory, mounted it in /etc/firejail. I'm happy.

That seems like the best solution honestly, it's much more portable and
flexible than changing the packaging of one or more packages.

<!-- gh-comment-id:1774184914 --> @kmk3 commented on GitHub (Oct 22, 2023): @birdie-github [on Oct 5](https://github.com/netblue30/firejail/issues/6033#issuecomment-1748954303): > > Currently each profile is an individual plain text file, which makes them easy > > to find, inspect, compare and manipulate in general using standard unix > > utilities. > > A zip file with profiles does not change that. > > > For example, comparing foo.profile to foo.local in any text editor. > > unzip exists. Currently I can go into /etc/firejail, open any profile in any text editor and quickly comment/uncomment lines while debugging a profile. I can do this multiple times in a row, at any time and without effort and be sure that the changes will be persistent. When I ask a user (who might not be very familiar with the CLI) to edit a profile in /etc/firejail for debugging, they are free to do so in their text editor of choice and they usually manage to accomplish it without issues. How would that work with unzip? I don't see how this would be possible without a significant impact to usability and accessibility. @birdie-github on [Oct 5](https://github.com/netblue30/firejail/issues/6033#issuecomment-1749264370): > Again, I wasn't talking about own firejail files, I was talking about the > free space fragmentation caused by having > 1200 small files all over your > partition. > > Secondly a directory with so many files can become fragmented itself. Since /etc is usually mostly consisted of small files, one option would be to use a small separate filesystem for it with smaller block sizes. > Anyways, I've solved this issue for myself using `fuse-zip`. Zipped the whole > directory, mounted it in `/etc/firejail`. I'm happy. That seems like the best solution honestly, it's much more portable and flexible than changing the packaging of one or more packages.
Author
Owner

@birdie-github commented on GitHub (Oct 23, 2023):

Currently I can go into /etc/firejail, open any profile in any text editor and
quickly comment/uncomment lines while debugging a profile.

Have profiles in /usr/share/firejail. If you want to edit one, you copy it to /etc or $HOME.

Since /etc is usually mostly consisted of small files, one option would be to
use a small separate filesystem for it with smaller block sizes.

I have no stamina to prove my point. OK, it's a bad idea, it's totally fine for a daemon to have thousands of config files and causing free space fragmentation.

That seems like the best solution honestly, it's much more portable and flexible than changing the packaging of one or more packages.

It's totally ad-hoc and I wouldn't want to deal with this.

You are free to reopen this issue once you hit three thousand profiles which all must be stored in /etc/firejail.

<!-- gh-comment-id:1775651945 --> @birdie-github commented on GitHub (Oct 23, 2023): > Currently I can go into /etc/firejail, open any profile in any text editor and quickly comment/uncomment lines while debugging a profile. Have profiles in /usr/share/firejail. If you want to edit one, you copy it to /etc or $HOME. > Since /etc is usually mostly consisted of small files, one option would be to use a small separate filesystem for it with smaller block sizes. I have no stamina to prove my point. OK, it's a bad idea, it's totally fine for a daemon to have thousands of config files and causing free space fragmentation. > That seems like the best solution honestly, it's much more portable and flexible than changing the packaging of one or more packages. It's totally ad-hoc and I wouldn't want to deal with this. You are free to reopen this issue once you hit three thousand profiles which all must be stored in /etc/firejail.
Author
Owner

@kmk3 commented on GitHub (Oct 23, 2023):

@birdie-github on Oct 23:

Currently I can go into /etc/firejail, open any profile in any text editor
and quickly comment/uncomment lines while debugging a profile.

Have profiles in /usr/share/firejail. If you want to edit one, you copy it to
/etc or $HOME.

I have no stamina to prove my point. OK, it's a bad idea, it's totally fine
for a daemon to have thousands of config files and causing free space
fragmentation.

You are free to reopen this issue once you hit three thousand profiles which
all must be stored in /etc/firejail.

If the issue is disk fragmentation, why would it matter whether the profiles
are stored in /etc or /usr/share?

<!-- gh-comment-id:1775775249 --> @kmk3 commented on GitHub (Oct 23, 2023): @birdie-github [on Oct 23](https://github.com/netblue30/firejail/issues/6033#issuecomment-1775651945): > > Currently I can go into /etc/firejail, open any profile in any text editor > > and quickly comment/uncomment lines while debugging a profile. > > Have profiles in /usr/share/firejail. If you want to edit one, you copy it to > /etc or $HOME. > I have no stamina to prove my point. OK, it's a bad idea, it's totally fine > for a daemon to have thousands of config files and causing free space > fragmentation. > You are free to reopen this issue once you hit three thousand profiles which > all must be stored in /etc/firejail. If the issue is disk fragmentation, why would it matter whether the profiles are stored in /etc or /usr/share?
Author
Owner

@kmk3 commented on GitHub (Nov 1, 2023):

(Feel free to continue the thread, I'm just updating the issue to reflect the
current status)

<!-- gh-comment-id:1788836602 --> @kmk3 commented on GitHub (Nov 1, 2023): (Feel free to continue the thread, I'm just updating the issue to reflect the current status)
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/firejail#3161
No description provided.