Azure BLOB storage with Dynamics 365 FnO X++

Part-1: Azure BLOB storage with Dynamics 365 for operations

25 May 2020|Azure and Dynamics

With the introduction of cloud based applications, Dynamics has opened doors for seamless integration with Azure services. This blog is created to portray one such example of how Azure capability can be utilized within Dynamics 365 for Operations.

Lets consider a common scenario where Dynamics 365 for Operations needs exchange of files between different applications. This can be a file produced by your .Net application (or) it can be a file produced by you production system,etc,. In Dynamics AX 2012 this situation was handled by creating common folder and sharing in network path where different applications can place their files. Upon introduction of D365, local paths were not accessible by Dynamics production URL so we need to rely on common internet service for accessing the file.

One of the convenient way to handle this situation is by using Microsoft Azure BLOB storage as the central repository to exchange files between different external systems. Using Azure BLOB as storage space now Dynamics 365 and other external applications can send files or consume files. Since we have the files in Azure, this can be easily accessible by Cloud API's provided by Azure.

Part-1: Creating Azure BLOB and establishing connection within Dynamics 365 for Operations

This blog series is created as 3-part series to explain how Dynamics 365 for Operations X++ code can be utilized to perform different operations with Azure blob files:

Part-1: Creating Azure BLOB and establishing connection within Dynamics 365 for Operations

Part-2: Consuming files stored in the Azure Blob Storage within Dynamics 365 Finance and Operations using X++ code

     a. Get list of file names stored in the blob.

     b. Get the memory stream of each files stored in the blob storage.

     c. Read the content of the file line by line.

Part-3: Azure Blob Upload file, Move file and Delete file in Dynamics 365 Finance and Operations using X++ code

---

Part-1: Creating Azure BLOB and establishing connection within Dynamics 365 for Operations

Before we proceed, we must make sure the following details are created.

1) Create Azure Block BLOB Storage account and blob container in Azure portal

Please refer https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portalfor more details.

In our example, below is the Blob created.                    

     Storage account name: sccblobstorage

     Blob container name: sccblobcontainer

2) Store the connection string in D365 FNO

Your application needs to access the connection string at runtime to authorize requests made to Azure Storage and so you can store the connection string in D365 FNO environment variable (e.g. A field in a parameter form).

The format of the connection string is

DefaultEndpointsProtocol=[http|https];AccountName=myAccountName;AccountKey=myAccountKey

Indicate whether you want to connect to the storage account through HTTPS (recommended) or HTTP, replace “myAccountName” with the name of your storage account, and replace “myAccountKey” with your account access key.

The connection string for the storage account can be found in the Azure portal.  

3) Blob container name

Note down the container name where the files are stored. In our example, the container name is “sccblobcontainer”

4) Store the Blob file path in D365 FNO

The blob container has a folder like structure and files has to be accessed in certain path. In order to access the file at particular location, the file path is required and stored in the D365 FNO environment variable. Using this file path, you can iterate the blob container to particular folder and access the files.

File path can be stored for below example as: “SainaCloudConsulting\Test1”

File name : TestFile.txt

---

Establish connection with the blob container

In Dynamics 365 for Operations, connection to the BLOB can be established with the help of following class libraries: 

• CloudBlobClient - Establishes client usage of Azure BLOB storage
• CloudBlobContainer - Enables access to created container
• CloudStorageAccount - Establishes connection to cloud storage account 

Here is the complete code for getting BLOB connected using Dynamics 365 for Operations:

1. CloudBlobClient  cloudBlobClient;
2. CloudBlobContainer  cloudBlobContainer;
3. CloudStorageAccount  cloudStorageAccount;
4. cloudStorageAccount  = CloudStorageAccount::Parse(“DefaultEndpointsProtocol=https;AccountName=sccblobstorage;AccountKey=K/X1oQfY30ZmYnl9s89545409jhtvnkbCvJzX7HWa+Bg==;EndpointSuffix=core.windows.net”);
5. cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
6. cloudBlobContainer  = cloudBlobClient.GetContainerReference(“sccblobcontainer”);

Part-2: Azure BLOB storage with Dynamics 365 for operations

25 May 2020|Azure and Dynamics

Consuming files stored in the Azure Blob Storage within Dynamics 365 Finance and Operations using X++ code 

Lets consider a scenario where third party business application uses Azure Blob as central repository for storing the files, which in turn needs to be consumed by Dynamics 365 finance and operations. 

Following standard .Net libraries helps to consume those files:

• using Microsoft.WindowsAzure.Storage;
• using Microsoft.WindowsAzure.Storage.Blob;
• using System.IO;

Part-2: Consuming files stored in the Azure Blob Storage within Dynamics 365 Finance and Operations using X++ code

Using the container connection created in Part-1 (Establish connection with the blob container) we can perform some basic operations with blob folder, as part of this blog we cover following area:

1. Get file name list
2. Get file memory stream
3. Read the content of the file line by line
4. Read the field value from the line

---

1) Get File Name List

This section describes how to retrieve list of file name present in the blob folder using standard classes:

• CloudBlobDirectory - Used to communicate with the blob folder
• IEnumerable - Used here to store list of file names retrieved
• IListBlobItem - Represents the list of blob items accessed
• CloudBlockBlob -  Represents a blob that is uploaded as a set of blocks 

Using the connection established in previous section, following code will fetch the list of files names from the cloud blob directory:

1. CloudBlobDirectory  cloudBlobDirectory;// The directory of the blob container
2. container  con;
3. cloudBlobDirectory = cloudBlobContainer.GetDirectoryReference(“SainaCloudConsulting\Test1”);//File path where the files are stored
4. System.Collections.IEnumerable lstEnumarable = cloudBlobDirectory.ListBlobs(false, 0, null, null);
5. System.Collections.IEnumerator lstEnumarator = lstEnumarable.GetEnumerator();
6. List filenames = new List(Types::String); 
7. while(lstEnumarator.MoveNext())
8. {
9.      IListBlobItem item = lstEnumarator.Current; 
10.      if(item is CloudBlockBlob)
11.      {
12.           CloudBlockBlob blob = item;
13.            blob.FetchAttributes(null, null, null);
14.            con = str2con(blob.name, "/");
15.            filenames.addStart(conPeek(con,conlen(con)));
16.      }
17. }

---

2) Get file memory stream

After retrieving the file names list, iterate the file list to get each file names, and get the file memory stream content.

1. CloudBlobDirectory  cloudBlobDirectory = _blobContainer.GetDirectoryReference("SainaCloudConsulting\Test1”);
2. CloudBlockBlob  blob = cloudBlobDirectory.GetBlockBlobReference(“TestFile.txt”);
3. System.IO.Stream  memory = blob.OpenRead(null,null,null);

---

3) Read the content of the file line by line

Using the .Net standard IO class libraries, contents of the file can be read.

1. System.IO.StreamReader streamReader = new System.IO.StreamReader(memory);
2. Str   strRecord = streamReader.ReadLine(); //read each line in the file

---

4) Read the field value from the line

Assuming the data is separated using comma field delimiter, we store each field data into container variable as shown below:

1. while(!System.String::IsNullOrEmpty(strRecord))
2. { 
3.      try
4.      {
5.           Container conRecord = str2con_RU(strRecord, ‘,');// Here the file delimiter is ‘,’.
6.           conPeek(conRecord, 1);// read first field data in the file line
7.           conPeek(conRecord, 2);// read second field data
8.      }
9. }

This blog has covered the list of operations used to read the files from the blob container and perform necessary business action in Dynamics 365 Finance and Operation.

 

Part-3: Azure BLOB storage with Dynamics 365 for operations

25 May 2020|Azure and Dynamics

Azure Blob Upload file, Move file and Delete file in Dynamics 365 Finance and Operations using X++ code 

Lets consider a scenario where file gets generated by Dynamics 365 for Operations, which in turn needs to be consumed by third party applications. We use following standard .Net libraries helps to create files within Dynamics and write the files to Azure Blob storage:

• using Microsoft.WindowsAzure.Storage;
• using Microsoft.WindowsAzure.Storage.Blob;
• using System.IO;

Part-3: Place the files in Azure blob storage that are generated in Dynamics 365 Finance and Operations using X++ code

Using the container connection created in Part-1 (Establish connection with the blob container) we can perform some write operations to the blob storage, as part of this blog we cover following area:

1. Upload stream to Blob
2. Move files between blob/Delete

---

1) Upload stream to Blob

Convert the stream generated in D365 FNO to a file and then store it into blob storage as below:

1. CloudBlobDirectory  cloudBlobDirectory;
2. CloudBlockBlob  cloudBlobReference;
3. cloudBlobDirectory  = blobContainer.GetDirectoryReference(“SainaCloudConsulting\Test1”);//File path where the files are to be stored
4. blobReference = blobDirectory.GetBlockBlobReference(“SainaCloudConsulting.txt”);
5. blobReference.UploadFromStream(stream, null, null, null); // Pass the stream in this method  

---

2) Move files between Blob folder (or) Delete file in Blob folder

Consider an integration scenario where you files are placed in one folder, after successful execution either you need to move it to different folder or delete the file from blob. This section handles code corresponds to moving files between blob folders or deleting files in blob folder

To cater this scenario, we established the blob connections separately for source and destination blobs. Using both of these instances, we iterate to the files path, access the files and then delete/upload the files to respective blob as shown below:

In below example, "TestFile.txt" file is stored in the Test1 folder and moved to Test2 folder and then removed from Test1 folder.

1. CloudBlobDirectory sourceCloudBlobDirectory = sourceBlobContainer.GetDirectoryReference("SainaCloudConsulting\Test1");
2. CloudBlobDirectory destinationCloudBlobDirectory = destinationBlobContainer.GetDirectoryReference("SainaCloudConsulting\Test2”);
3. CloudBlockBlob SourceBlob = sourceCloudBlobDirectory.GetBlockBlobReference("TestFile.txt”);
4. CloudBlockBlob destinationBlob = destCloudBlobDirectory.GetBlockBlobReference(“TestFile.txt”);

Now if you wish to move the file between folders, then user following code:

5.destinationBlob.UploadFromStream(sourceBlob.OpenRead(null,null,null),null,null,null);

After processing if you wish to delete the file from blob, then user following code:

5.sourceBlob.Delete(0,null,null,null); // to delete the file inside blob 

Thanks for your time reading the 3-part series of accessing Azure container files. Hope this series would have helped you to add additional knowledge toward extended Azure capabilities from Dynamics 365 for Operations.