New post
AllAccepted answers
Robert Cemper · Sep 3, 2020

let's have  your  binary  input in variable data 

JAVA                                  ObjectScript
byte[] data = new byte[1024];                                            ;
Bytebuf buf  = new ByteBuf(data)      set buf=data              ; make your work copy  
byte type = buf.read()                set type=$extract(buf)    ; if type = character 
                                      set type=$ascii(buf)      ; if type = binary
int len = buf.ReadInt()               set len=$zlascii($extract(buf,2,5))           ; if order low->high
                                      set len=$zlascii($reverse($extract(buf,2,5))) ; li order high->low 
byte[] bodyBuff = new byte[len]       ; 
buf.Read(bodyBuff)                    ; 
String str = new String(strBody)      set str=$extract(buf,6,*) ; take rest of buf


from the description there are 2 open points:

  • is type a binary value or a character   
  • for length it could be
  • -         low byte first   so 515  => x/02 03 00 00  ;  default
  • -  or high byte first  so  515 => x/00 00 02 03   ;  need to reverse order

therefore I have added both variants

Robert Cemper · Sep 2, 2020

That's interesting: 
How did you look for GBLOCKCOPY.int  ?  there never was a  .mac!
And it is even on the news IRIS.

Instance: IRIS
USER>zn "%SYS"
%SYS>zl GBLOCKCOPY
%SYS>p
GBLOCKCOPY ; Fast global copy from database to namespace
         ; Copyright (c) 2020 by InterSystems Corporation.
         ; Cambridge, Massachusetts, U.S.A.  All rights reserved.
         ; Confidential property of InterSystems Corporation.
         n
         i $p($g(^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")),"~",1)'="1.0" k ^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")
         i '$d(^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")) s ^|"^^"_$zu(12)|%SYS("GBLOCKCOPY")="1.0~"
         s POP=0
         w !,"This routine will do a fast global copy from a database to another database or"
         w !,"to a namespace. If a namespace is the destination, the global will follow any"
         w !,"mappings set up for the namespace."
         ;
MENU     ;

If this is missing you may also miss other important pieces:
Check your installation or reinstall it

Robert Cemper · Sep 2, 2020

This is a late reply and the content my explain why.
There is a set of available classes and methods to fulfill your requirements.
Though you have to be careful as there are side conditions to be considered that require special attention.

You have to wrap these class methods in some routine or class and then plug it into the standard task manager.

From your question I understand 4 different tasks:

  • Compact globals in a database   (=force global to make better use of data blocks) ##Class(SYS.Database). Compact(..............)
  • Return unused space for a database (=truncate DB) ##class(SYS.Database).ReturnUnusedSpace(........)
  • Defragment a database (=make Blocks contiguous)  ##class(SYS.Database).Defragment(........) ##class(SYS.Database.BackgroundDefragment).Start(........)  ;;[not so useful for task manager]
  • Compact free space in a database (=really returns disk space) ##class(SYS.Database).FileCompact.(........) ##class(SYS.Database.BackgroundFileCompact).Start(........)  ;;[not so useful for task manager]

The official documetation also has some examples.
My personal favorite is routine DATABASE.int in namespace %SYS
It covers all these functions for interactive use and illustrates also all possible requirements and limits in detail.
The labels you should check are:
COMPACT()
DBTruncat()
DEFRAG()
FileCompact()

This might also explain why there is not just a method out of the box.

Robert Cemper · Sep 2, 2020

Dear @Sylvain Guilbaud 
As this is the oldest unanswered question I found I'll give it a try:
One of the most attractive features of DeepSee is its close link to the Caché (now IRIS) database using 
the DSTIME mechanics for updates.
With an external linked DB you don't have this "built-in" support.
Other DB's have similar features but including them requires some "art work".
And that doesn't seem to happen so often.

On the other hand the question was never expired nor closed.
It might happen that new results (almost 5 years later) show up now as it moves to the front.

J'espère que tu vas bien et en bonne santé!  yes
Salutations de Vienne,

Robert (toujours actif) wink

Robert Cemper · Sep 2, 2020

@Oliver Wilms 
#1) make  sure your ECP Server is willing to accept ECP AplicationServers: 

#2) on your Application Server define your Data Server

%SYS>set Name="MainServer"
%SYS>set Properties("Address")="127.0.0.1" 
%SYS>set Properties("Port")=1972
%SYS>s Status=##Class(Config.ECPServers).Create(Name,.Properties)
%SYS>zw Status
Status=1

see docs: class Config.ECPServers

#3) connect to your Data Server

%SYS>set ecpstate=##class(SYS.ECP).GetServerConnState(Name)
%SYS>zw ecpstate
ecpstate=1      ;; Not Connected
%SYS>set tSC=##class(SYS.ECP).ServerAction(Name,3)   ;; 3 -  Change to Normal
 %SYS>zw tSC
tSC=1
%SYS>set ecpstate3=##CLASS(SYS.ECP).GetServerConnState(Name)
%SYS>ZW ecpstate3
ecpstate3=5   ;; 5 - Normal

see docs: class SYS.ECP

#4) now in SMP:

Robert Cemper · Sep 1, 2020

4GB is a significant amount.
A different approach could be to have just an additional DB  (not journaled)
where instead of a loop you use MERGE to copy the global
MERGE ^|"newdb"| myGLOBAL = ^myGLOBAL
once your collection is done you dismount the deb and just copy it  wherever you need it. 
"newdb" is the extended Global reference. Either just the DB or a helping namespace.

So you can check exactly what data you move.

Of course, this works also across ECP if it is available.

Robert Cemper · Aug 31, 2020

make sure your / is really just a / and not surrounded by some nonprinting stuff
do a check e.g on https://www.xmlvalidation.com/ 

(ending at line 1 character 183)  could also mean you are missing
<?xml version="1.0" encoding="UTF-8"?>  as first line.

next possible trap: your xml is not encode in UTF-8  (but in some other [windows?]
code after modification with Notepad.exe or similar.

Robert Cemper · Aug 31, 2020

very quick:  $DATA() returns:

  • 0 no node, no data
  • 1 node exist and has data
  • 10 node exist has descendants but no data
  • 11  node exist has descendants and has data

# is the modulo operator and #10 means you just get the rightmost part of the $Data() result : 1 or 0 (true/false)
So $Data(^EDLIST("DIAG",code),value)#10 means: If the node has data than the content is in  value.
otherwise value is useless, skip it

Robert Cemper · Aug 29, 2020

A test with 102,794 rows confirmed the performance difference.
#1)  on index xitm
Rows selected: 61320 Global Ref: 488,861 Commands: 5.244,585 Cached Query: %sqlcq.CACHE.cls16

#2) on index ycol
Rows selected: 61320 Global-Ref: 133,812 Commands: 3.909,205 Cached Query: %sqlcq.CACHE.cls13 

Reducing the query output to SELECT COUNT(ID) makes the difference even more dramatic
#1) Performance: 0.457 sec  Global-Ref: 574,193 Commands: 2.138,695
#2) Performance: 0.082 sec  Global-Ref: 205,973 Commands:   724,288