New post
Question
· Jul 13, 2023

Embedded Python and IRIS Streams

#Embedded Python #Interoperability #ObjectScript #Health Connect #InterSystems IRIS #InterSystems IRIS for Health

Hi All,

We're doing our first babysteps with embedded Python and IRIS with an interoperability solution. We want to convert a CSV file to an Excel xlt file.

As service we've got a EnsLib.File.PassthroughService which picks up a csv file

As operation we've got a EnsLib.File.PassthroughOperation which writes the Excel file.

In the middle:

We've created an Business Process with an %Stream.GlobalCharacter in the Request and a %Stream.GlobalCharacter  in the Response.

But how do we get the %Stream.GlobalCharacter in the Python ClassMethod? Ideally I don't would like to write the stream first to a temp file, give the temp file to Python and let Python write the XLT file and read that XLT file into a %Stream.GlobalCharacter.

Any ideas? See the code snipped below. This one works but will first create a string from the Stream but will fail when the csv is bigger then the Intersystems string limit

Class ZORG.BP.CsvToExcel Extends Ens.BusinessProcess
{

Method OnRequest(pRequest As ZORG.BP.CsvToExcel.CsvToExcelReq, Output pResponse As ZORG.BP.CsvToExcel.CsvToExcelResp) As %Status
{
    Set tsc = $$$OK
	; Create the response class
	Set pResponse = ##class(ZORG.BP.CsvToExcel.CsvToExcelResp).%New()
	Set pResponse.outputStream = ##class(%Stream.GlobalCharacter).%New()
	
	$$$TRACE("Size of Stream: "_pRequest.inputStream.SizeGet())
    Set XlsString=..convertCSVtoXLS(pRequest.inputStream.Read($$$MaxStringLength))
	Do pResponse.outputStream.Write(XlsString)
	
	return tsc
}

ClassMethod convertCSVtoXLS(csvFile As %String(MAXLEN="")) As %String [ Language = python ]
{

	"""
    This function takes a CSV string as input and converts it into an XLS string.
    
    Parameters:
    csv_string (str): The CSV string to be converted
    
    Returns:
    str: Th
	"""
	import pandas
	import xlwt
	import sys
	import csv
	import tempfile
	from io import StringIO
	
	# Read the CSVdata into a Pandas DataFrame
	
	df = pandas.read_csv(StringIO(csvFile), delimiter=";", decimal=",", dtype={'patno':str} )	
	# Create an instance of Workbook class from xlwt library	
	workbook = xlwt.Workbook()

    # Add a sheet to the workbook
	sheet = workbook.add_sheet('Sheet1')

    # Write the DataFrame values to the sheet
	for i, column in enumerate(df.columns):
		sheet.write(0, i, column)  # Write column headers
		for j, value in enumerate(df[column]):
			if ( str(value) != 'nan' ):       #catch empty cells
				sheet.write(j + 1, i, value)  # Write cell values

	# Write workbook to temporary file
	file = tempfile.TemporaryFile()
	
	# Save Workbook
	workbook.save(file)
	
	#Go to the beginning of the file
	file.seek(0)

	#Read from temporary File and return
	data = file.read()
	return(data)
}
}

ObjectScript
ObjectScript
Product version: IRIS 2021.2
$ZV: IRIS for Windows (x86-64) 2021.2 (Build 649U) Thu Jan 20 2022 08:46:32 EST [HealthConnect:3.4.0] [HealthConnect:3.4.0]
Discussion (3)1
Log in or sign up to continue
Guillaume Rongier · Jul 13, 2023

That how i read Stream and Write stream with Embedded Python :

Read Stream :

def stream_to_string(stream)-> str:
    string = ""
    stream.Rewind()
    while not stream.AtEnd:
        string += stream.Read(1024)
    return string
Python
Python

Write Stream :

def string_to_stream(string:str):
    stream = iris.cls('%Stream.GlobalCharacter')._New()
    n = 1024
    chunks = [string[i:i+n] for i in range(0, len(string), n)]
    for chunk in chunks:
        stream.Write(chunk)
    return stream
Python
Python
2 2
Menno Voerman · Aug 8, 2023

@Guillaume Rongier, I've had some struggles with converting but that was nog due to your functions. 

See a snippet of working code below:

ClassMethod Decrypt(Stream As %Stream.GlobalCharacter, key As %String) As %Stream.GlobalCharacter [ Language = python ]
{
    import iris
    import os
    from cryptography.hazmat.primitives import hashes, padding
    from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
    from cryptography.hazmat.backends import default_backend
    from base64 import b64decode

    def stream_to_string(stream)-> str:
                   string = ""
                   stream.Rewind()
                   while not stream.AtEnd:
                                   string += stream.Read(1024)
                   return string

    def string_to_stream(string:str):
                   stream = iris.cls('%Stream.GlobalCharacter')._New()
                   n = 1024
                   chunks = [string[i:i+n] for i in range(0, len(string), n)]
                   for chunk in chunks:
                                   stream.Write(chunk)
                    return stream

    # Convert the Base64 encoded key to bytes
    key_bytes = b64decode(key)

    cipher_bytes = bytes(stream_to_string(Stream),'iso-8859-1')

    # Extract the IV from the first 16 bytes of the cipher
    iv = cipher_bytes[:16]

    # Create the AES cipher object
    backend = default_backend()
    cipher = Cipher(algorithms.AES(key_bytes), modes.CBC(iv), backend=backend)
    decryptor = cipher.decryptor()

    # Decrypt the data, excluding the IV
    decrypted_bytes = decryptor.update(cipher_bytes[16:]) + decryptor.finalize()

    # Remove padding
    unpadder = padding.PKCS7(algorithms.AES.block_size).unpadder()
    decrypted_data = unpadder.update(decrypted_bytes) + unpadder.finalize()
    
    return string_to_stream(decrypted_data)
  }
ObjectScript
ObjectScript
1 0