Deflate compression
During the realization of some required functionality, I discovered that I need to use Deflate compression, and found, that IRIS does not offer any options, $system.Util.Compress, works one way open (GZIP/COMPRESS) totally different, even for the same algorithms, I see that what I would expect as a result for Deflate somewhere there, but Compress returns something more. Is there any way to get a correct result using just ObjectScript without a need to use external tools? Was it really so difficult to have Deflate implemented in the first place?
set original = "my very long string which needs to be deflated"
zwrite original
write !,"using $system.Util.Compress",!
set compressed = $system.Util.Compress(original, "zlib")
zwrite compressed
zzdump compressed write !
set deflated = $system.Encryption.Base64Encode(compressed, 1)
zwrite deflated
set tmpFile = ##class(%File).TempFilename("bin")
set io = $io
open tmpFile:("WNS":::/COMPRESS="zlib":/GZIP=0):0
use tmpFile
write original
use io
close tmpFile
set stream = ##class(%Stream.FileBinary).%New()
set stream.Filename = tmpFile
set stream.RemoveOnClose = 1
set compressed = stream.Read(stream.Size)
write !,"using open file",!
zwrite compressed
zzdump compressed write !
set deflated = $system.Encryption.Base64Encode(compressed, 1)
zwrite deflated
write !,"Expected",!
set base64 = "y61UKEstqlTIyc9LVyguKcoEUuUZmckZCnmpqSnFCiX5CkmpCimpaTmJJakpAA=="
set compressed = $system.Encryption.Base64Decode(base64)
zwrite compressed
zzdump compressed write !ObjectScriptObjectScriptWhy the results are different, and how can I make it understandable by not only IRIS?
.png)
What is the point of having a compress function, which says that it uses some standards but only IRIS can decompress?
.png)
But results I got from IRIS
.png)
ZLIB using OPEN command recognized as GZIP, not a Deflate anyway
.png)
I've been there before on IRIS zlib compression library....
The problem is explained in Compress(), from Class Reference %SYSTEM.Util, Compress() method (emphasis mine):
Returns the compressed form of a given string.
When Type is specified, it is the compress algorithm used to compress the data. Its value can be "zlib", "zstd" or "lz4". If Type is not specified then the "zlib" compression algorithm is used. A compressed string is a compressed data block with a customized wrapper. Use Decompress() to decompress it.
So, after Compress() you need to remove the "customized wrapper":
set compressed=$e(compressed,4,*-5)Unfortunately Decompress() needs the custom wrapper, and I don't know how to make it up, however, as in your sample, I/O can be used, using file in your sample.
Regarding the file I/O, note that form Open command documentation:
"zlib" — Use the zlib compression library. /COMPRESS="zlib" is equivalent to /GZIP=1
So, if you change:
open tmpFile:("WNS":::/COMPRESS="zlib":/GZIP=0):0With:
open tmpFile:("WNS":::/GZIP=0):0It will then work just fine.
I fully agree that it's a pity to have zilib compress/uncompress utility "custimized" with proprietary wrapper that makes it unusable for any "standard use". However in that case you can use I/O, file or other devices. Personally I use XDEV for compress/decompress and leave the silly %SYSTEM.Util.Compress()/Decompress() alone.
The way to remove the wrapper from the Compress method, did not help. It's still not recognizable.
Only file way, and removing the first 11 bytes of GZIP header worked for me.
But I suppose, it's just luck. With different data and size of that data, it may not work.
The resulting archive is easily recognized/unpacked in WinRAR, 7z:
s f=##class(%Stream.FileCharacterGzip).%New() s f.TranslateTable="UTF8" s f.Filename="C:\test.gz" d f.WriteLine("test") d f.Write("Привет Caché") d f.%Save()I need DEFLATE not GZIP, gzip works fine
For DEFLATE use:
set original = "my very long string which needs to be deflated" zwrite original write !,"using $system.Util.Compress",! set compressed = $system.Util.Compress(original, "zlib") set compressed=$e(compressed,4,*-5) zwrite compressed zzdump compressed write ! set deflated = $system.Encryption.Base64Encode(compressed, 1) zwrite deflated set tmpFile = ##class(%File).TempFilename("bin") set io = $io open tmpFile:("WNS":::/COMPRESS="DEFLATE"):0 use tmpFile write original use io close tmpFile set stream = ##class(%Stream.FileBinary).%New() set stream.Filename = tmpFile set stream.RemoveOnClose = 1 set compressed = stream.Read(stream.Size) write !,"using open file",! zwrite compressed zzdump compressed write ! set deflated = $system.Encryption.Base64Encode(compressed, 1) zwrite deflated write !,"Expected",! set base64 = "y61UKEstqlTIyc9LVyguKcoEUuUZmckZCnmpqSnFCiX5CkmpCimpaTmJJakpAA==" set compressed = $system.Encryption.Base64Decode(base64) zwrite compressed zzdump compressed write !Result:
@Dmitry Maslennikov, does this address your question?
Yep, that works, thanks, pity that it was not implemented in Compress method, but at least something