CPU non-usage + Leopard support in v >= 1?

Talk about Keka
Forum rules
Talk about Keka here. For bugs go to Issues.
Post Reply
blacknova
Keka & Me
Keka & Me
Posts: 5
Joined: Sun Mar 04, 2012 4:08 pm

CPU non-usage + Leopard support in v >= 1?

Post by blacknova » Sun Mar 04, 2012 6:34 pm

Hi,
First off, thanks for writing Keka. Of the 2-3 7zip compression tools available for OSX, Keka is by far one of the better ones.

Secondly, I have a few questions regarding Keka v0.1.4.2 working with files on a MacbookPro3,1 (Core 2 Duo 2.4GHz/6GB RAM) + OSX v10.5.8:
  • Is it normal for Keka to not use any CPU at all?
    When I began decompressing and testing (t option) a 32 x 1GB (-m0) spanned archive on an NTFS v5 volume, it consistently only uses 1-3% of the CPU [1] even though the rest of the machine is idle. Normally if I were multitasking this would be a good thing but it'd be nice if the application used more resources if the machine was just sitting there idle.

    I looked into using "renice" and bumped up Keka, keka7z (and even ntfs-3g)'s priority to "51" [2], but it didn't seem to make any difference at all when I observed the CPU usage for keka* in Activity Monitor.app. I've heard priority control via (re)nice is broken in OSX v10.5.8+ though. If so, how can I speed up Keka?
  • Any idea when Leopard support will be added for v >= 1?
    I know we're up to Snow Leopard and now even Lion, but a lot of things (ex: rEFIt) break so I don't think I can upgrade at the moment. I ask because I've read that v1+ will now show you estimates of how long a compression / decompression ( / test?) run will take VS just the total elapsed time in the progress bar :).
Thanks for any help you can provide.

Regards,
-B


[1] Actually, Keka.app used up to 3% CPU and keka7z would normally only use ~1%, even after re-niceing
[2] I'm assuming higher values = higher priority based on the fact that things like daemons and SystemUIServer have values > 31, but I tried going in both directions anyway with no visible performance changes.
User avatar
aone
Mr. Keka
Mr. Keka
Posts: 265
Joined: Sun Feb 26, 2012 8:42 pm
Contact:

Re: CPU non-usage + Leopard support in v >= 1?

Post by aone » Sun Mar 04, 2012 8:33 pm

About the CPU ussage, I always try to use the less CPU I can with Keka, so p7zip, tar or unrar can use the rest.
I'm not an expert in p7zip-7-zip, but in all my tests I've seen that it depends on the files being compressed/extracted the binary uses more or less CPU. It will be interesting to see a manual decompression test of your 32x1GB (BTW huge file!) with p7zip or keka7z using the Terminal.app to see if there's any difference. Also, since you used the -m0 switch, this file is not compressed so this may be a hard-disk work, not an extraction+CPU work.

About the Leopard build, I had troubles compiling p7zip for 10.5. Now I'm in trouble compiling Keka 1.0 with PPC support. By now I have a working Intel-only version. I attach that version here so you can test (since your Mac is an Intel). Hope it works fine :)

Since I get lots of feedback and fixed lots of bugs, the Leopard build will be released in the 1.0.1 branch. No need to do a 1.0.
You do not have the required permissions to view the files attached to this post.
aone ~
blacknova
Keka & Me
Keka & Me
Posts: 5
Joined: Sun Mar 04, 2012 4:08 pm

Re: CPU non-usage + Leopard support in v >= 1?

Post by blacknova » Mon Mar 05, 2012 4:01 am

Hi aone,
Thanks for the quick response.
It will be interesting to see a manual decompression test of your 32x1GB (BTW huge file!) with p7zip or keka7z using the Terminal.app [...] Also, since you used the -m0 switch, this file is not compressed so this may be a hard-disk work, not an extraction+CPU work
So I had a hunch ( :idea: ) and decided to try creating a new (journaled) HFS+ partition and copy the files there to perform the extraction to see if the slowness was due to going through the NTFS-3G driver.

I re-ran the extraction with v0.1.4.2 (this time from the CLI [1]) with a reniced priority of 51 and ... low and behold the extraction completed in ~45 mins! (VS the ~5 hours or so through the NTFS-3G (v2011.4.12 + FUSE 27) drivered partition ) !! :D

Note: the CPU usage was still pretty low (although slightly higher with peaks around ~6%), but I think you are right that the low CPU-usage is due to the activity being more IO-bound than CPU-bound. A quick look at my iStat monitors during the NTFS-3G VS OSX HFS+ tests probably confirms this as it showed periodic 2.25MB/s burst writes in the former VS 10~13MB/s sustained RWs in the latter.

However...
The only problem now seems to be that the item extracted is a single, monolithic ~30GB file VS the original folder with files that I had put in :|

I tried a few other things like doing an extract (VS extract), toggling the "Archive as single files" option (not completely clear on what this does), and redo-ing the extraction with the new Leopard build you attached, but nothing seemed to help. According to the -t option the archive isn't corrupted either so I don't know what's wrong. :(

Any ideas how I can recover my original files?

-B



[1] Ironically, I actually created shell scripts to run it from the CLI several months ago when I first installed Keka v0.1.4.2 because I wanted to see a progress indicator but I ended up not using them when I realized that the CLI output didn't offer any progress indicator either, lol (I suspect this may be more of an issue with the p7zip binary though)
User avatar
aone
Mr. Keka
Mr. Keka
Posts: 265
Joined: Sun Feb 26, 2012 8:42 pm
Contact:

Re: CPU non-usage + Leopard support in v >= 1?

Post by aone » Mon Mar 05, 2012 12:23 pm

Good to know the first part. NTFS-3G is good for compatibility but not for performance :) Sure the paid Tuxera version has improved a lot.

Archive as single files is only for compression. For example, if you drag and drop two .txt files with this option activated Keka will create two compressed files, one per each .txt. For the record. I must update the documentation, but I have no time!!! :?

My mind can't understand why extracting this on a NTFS volume has no problems but the same operation (with the same Keka 0.1.4.2 if I'm not lost) in a HFS volume has some troubles... The 30x1GB was any kind of tarball? You can make a -l to see it's contents. My first thought was this is a tarball and the extracted tar lacks it's extension. So you can try to extract this file (with the 1.0 version just drop it to the extract zone). You also can make an ls -la in the terminal to see if there's any problem with your permissions or if there's any .Keka-XXX hidden folder.

It would be helpful to have your file to test it, but hey, don't attach me a 30GB file! :lol:
aone ~
blacknova
Keka & Me
Keka & Me
Posts: 5
Joined: Sun Mar 04, 2012 4:08 pm

Re: CPU non-usage + Leopard support in v >= 1?

Post by blacknova » Mon Mar 05, 2012 7:33 pm

Archive as single files is only for compression. For example, if you drag and drop two .txt files with this option activated Keka will create two compressed files, one per each .txt.
Ahh, ok.
The 30x1GB was any kind of tarball? You can make a -l to see it's contents.
When I do a "keka7z -l .." on the original archive, it and only lists one file. When I originally created the archive I added a password and encrypted the file names though, so I'm not sure if that would that make a difference.
My first thought was this is a tarball and the extracted tar lacks it's extension. So you can try to extract this file (with the 1.0 version just drop it to the extract zone). You also can make an ls -la in the terminal to see if there's any problem with your permissions or if there's any .Keka-XXX hidden folder.
I thought something similar and dropped the resulting output file back into Keka but it just prompts for a password :?: . Regardless of what password I enter (including the one used to create the original archive), it just keeps reporting it is incorrect :?: (I tested this several times to make sure I wasn't typing it incorrectly), so I'm not sure what this monolithic file is... (*NIX file command reports it is a 7-zip file regardless if I rename it to .tar and tar doesn't recognize it as a .tar file either).

I'm not sure what you mean by a "permission problem"...

During extraction w/ v1.0Leo there is a .Keka-XXX folder but that seems to disappear once the extraction completes successfully, which it seems to be.
My mind can't understand why extracting this on a NTFS volume has no problems but the same operation (with the same Keka 0.1.4.2 if I'm not lost) in a HFS volume has some troubles...
The issue happens regardless of the underlying FS this is extracted on.

To rule out NTFS-3G playing a part at all, I copied over the original files and re-compressed (or "stored") it there with v1.0Leo (it only took ~50 min BTW) . When it finished I tried extracting it but about halfway through it errored out with

Code: Select all

Extraction of "..." failed.  Error code 334 using "p7zip"  Bad password try again
?!? :x which makes no sense at all because it accepted the password when I first started the extraction. I have a feeling this may be due to the HD running out of space, so I moved some files around and have kicked off the extraction again but we will see..

Starting to get frustrated ... *sigh*
blacknova
Keka & Me
Keka & Me
Posts: 5
Joined: Sun Mar 04, 2012 4:08 pm

Re: CPU non-usage + Leopard support in v >= 1?

Post by blacknova » Tue Mar 06, 2012 3:38 am

To rule out NTFS-3G playing a part at all, I copied over the original files and re-compressed (or "stored") it there with v1.0Leo (it only took ~50 min BTW) . When it finished I tried extracting it but about halfway through it errored out with
Clarification: copied over the original files = copied the original files over to the HFS+ partition.

----
Update: It looks like this time it succeeded.. *phew!*, so maybe it was the space after all but:
  • the error message I got was definitely deceptive, and
  • I am still upset that I spent a few days compressing, uploading, downloading, and extracting a seeming non-corrupt 32 x 1GB v0.1.4.2 archive only to find out it effectively just "transformed" my folder of files into a single, monolithic, 30GB, mystery file, lol. Thank *god* I didn't delete the originals....
User avatar
aone
Mr. Keka
Mr. Keka
Posts: 265
Joined: Sun Feb 26, 2012 8:42 pm
Contact:

Re: CPU non-usage + Leopard support in v >= 1?

Post by aone » Tue Mar 06, 2012 10:54 pm

Sorry about all the mess. But you succeed then?

I must fix the error code 334 appearing after a good password was entered, or enhance it with two error codes, the 334 and the binary error code. I've created the 334 error cause p7zip uses error code 2, and this is a very generic error.
aone ~
Post Reply