DeflOpt thread

[Zardalu]

.

I created V2.01, but unfortunately I can't get to it right now. As soon as I have retrieved it and have run it against a batch of test files without any problems, I'll release it.

Would it be also possible to add a logging option so that the output is redirected to a file?

Yes, it's possible, but why not add something like ">>c:\output.log" to the command line?

I also run into an

[fred01]

Would it be also possible to add a logging option so that the output is redirected to a file?

Yes, it's possible, but why not add something like ">>c:\output.log" to the command line?

I tried to put this it in zipmax config sometime ago, but it would not work, so used a batch file as a temporary solution for now, but was hoping to have a more sophisticated way :-)
Thank you for the new version, will try it as soon as I'm back (writing from a cybercafe now).

[Zardalu]

I released V2.01. I also added a "silent" option.

http://www.walbeehm.com/download/

Ben Jos.

[Zardalu]

V2.02 released. Significantly faster than V2.01. Same performance in optimisation.

Ben Jos.

[Thundik81]

Thanks a lot, Ben

Quick comparison on png files (screenshots compressed with pngoutwin)

CPU : Bi-Xeon @ 3000
Timer for calculating execution time (in addition of native cycles counting)
3 successive runs for each version


DeflOpt V2.00

Code:

Number of files processed  :       59
Number of files rewritten  :       59
Total number of bytes saved:    7,081

39,272,848,147 cycles.
Kernel Time  =     0.109 = 00:00:00.109 =   0%
User Time    =    12.640 = 00:00:12.640 =  95%
Process Time =    12.750 = 00:00:12.750 =  96%
Global Time  =    13.172 = 00:00:13.172 = 100%

38,863,666,680 cycles.
Kernel Time  =     0.109 = 00:00:00.109 =   0%
User Time    =    12.609 = 00:00:12.609 =  96%
Process Time =    12.718 = 00:00:12.718 =  97%
Global Time  =    13.015 = 00:00:13.015 = 100%

38,891,161,350 cycles.
Kernel Time  =     0.156 = 00:00:00.156 =   1%
User Time    =    12.656 = 00:00:12.656 =  97%
Process Time =    12.812 = 00:00:12.812 =  98%
Global Time  =    13.015 = 00:00:13.015 = 100%

DeflOpt V2.02

Code:

Number of files processed  :       59
Number of files rewritten  :       59
Total number of bytes saved:    7,081

14,848,200,277 cycles.
Kernel Time  =     0.140 = 00:00:00.140 =   2%
User Time    =     4.671 = 00:00:04.671 =  94%
Process Time =     4.812 = 00:00:04.812 =  96%
Global Time  =     4.969 = 00:00:04.969 = 100%

15,374,020,950 cycles.
Kernel Time  =     0.109 = 00:00:00.109 =   2%
User Time    =     4.734 = 00:00:04.734 =  91%
Process Time =     4.843 = 00:00:04.843 =  93%
Global Time  =     5.156 = 00:00:05.156 = 100%

15,477,956,955 cycles.
Kernel Time  =     0.156 = 00:00:00.156 =   3%
User Time    =     4.703 = 00:00:04.703 =  90%
Process Time =     4.859 = 00:00:04.859 =  93%
Global Time  =     5.203 = 00:00:05.203 = 100%

[ace_dent]

Ben,
thanks for this useful tool to knock a few more bytes from my pngs. I'm going to be adding it to my PNG Optimization Guide soon and wondered if / what / when you had any updates planned. I remember a while ago you said there might be a few further enhancements down the road...

Thanks again,
Andrew

[Zardalu]

wondered if / what / when you had any updates planned. I remember a while ago you said there might be a few further enhancements down the road...

I have some ideas for looking at more trees, but I haven't yet come up with a way to efficiently implement it. I might try to do it in a slow way first to see if it is worth it. In addition, it turns out that a tree that looks better sometimes produces worse results. I'm thinking of looking at that too.

Either way, I don't know whether it will be tomorrow or next week or next month...

Ben Jos.

[ace_dent]

Ben,
thanks for the feedback. I just incorporated DeflOpt into my script using the new silent switch. Although this makes for a less verbose output, the program 'header' (*** DeflOpt ... *** , etc.) is still produced. Ideally no output would be generated in silent mode. If you would consider this change / enhancement, that'd be cool.
Many thanks,
Andrew

[secretpng]

Zardalu, any chance you could post it at your download site: http://www.walbeehm.com/download/ ?

I have recently tested DEFLOPT for the first time. It works great and was looking to test BJWFlate also.

Thank You,

[Zardalu]

Andrew: I'm considering it... and, better yet, I'll probably do it in the next version, but I've been a bit busy lately... real life and such... there are some things I like to do more in my free time than programming, although there are not many. Don't know what kind of smiley to end this with...

secretpng: Try Roman Scherzer's site:  http://www.clrmame.com/bjwflate.htm... it's very close to the latest... I don't think I've ever released 1.54x (an EVEN slower version that might give slightly better results) and 1.55 (which sometimes did better than 1.54 and sometimes worse), but, well, BJWFlate is OLD. It was decent enough when I released it, but the latest versions of Ken's KZIP beat it 99% of the time, while also being a lot faster. There MIGHT also be a very rare bug in BJWFlate... I'm not sure... someone reported some file that BJWFlate allegedly created a corrupt archive for, but I never was interested enough anymore to even check. Sorry. I don't think it's worth it for me to develop BJWFlate any longer. Ken's KZIP is so much better and faster in almost every case. Maybe at some point, I'll get interested again, but then I would write something completely new instead of making BJWFlate better at the cost of even more speed.

Ben Jos.

[fred01]

I tried DeflOpt (v2.02) on an OpenOffice.org files, but it failed with:

Code:

ERROR: CRC in local fileheader differs from the one in the central directory.

The only other program to fail on testing such files is AdvanceCOMPzip (v1.15), with:

Code:

Invalid crc on data descriptor

Other zip utilities, including 7zip and PKZIP (DOS and windows), test and extract these files correctly. But, peculiarly, PKZIPFIX (or -fix) would

[Zardalu]

The ZIP format has two directories. Local and glocal. Both should contain the same information. But I found that MP3s and lots of other files in these days of file sharing don't contain the last few bytes. PKZIPFIX only tries to reconstruct the global directory from the local ones. Please remember that PKZIPFIX is OLD.

Having said that, tell me where I can find the files you stated failed. Or where I can download the failures. I think DeflOpt isn't the problem, but, rather, it's a case of bad downloads. Still... if I can fix "easy" errors, I might consider adding them... .

Ben Jos.

[fred01]

Thank you for the reply.

The ZIP format has two directories. Local and glocal. Both should contain the same information. But I found that MP3s and lots of other files in these days of file sharing don't containt the last few bytes.

Yes, I suspected that it had something to do with local/global header discrepancy, but I'm not familiar enough with their binary content to see where. Is there an "intelligent" hex viewer that would show all the different parts of a ZIP header, like date, size, CRC, etc.?
I was not aware that data in MP3s were also compressed with ZIP.

PKFIX only tries to reconstruct the global directory from the local ones. Please remember that PKZIPFIX is OLD.

I also tried pkzipc -fix and it produced binary identical file with pkzipfix's. And it seems that the global directory is good, but

[Awesoken]

I was not aware that data in MP3s were also compressed with ZIP.

There is no ZIP in MP3. You read Zardalu's words wrong. He was referring to a bug with file sharing services not always sending the whole file. MP3, being a popular file type, is often where the problem is noticed.

[fred01]

There is no ZIP in MP3. You read Zardalu's words wrong. He was referring to a bug with file sharing services not always sending the whole file. MP3, being a popular file type, is often where the problem is noticed.

Ok, I get it. Thank you for clarification.