Copy large data objects from Azure BLOB to Amazon S3

Scenario

This article is going to focus on providing guidance and library for copying Azure BLOB to Amazon S3. Using .NET (C#), have created few classes which will help us to achieve copying large BLOB into Amazon S3.

Points to be considered

1. As we're targeting large data objects to be copied, we need perform Multipart upload. 
2. From Azure BLOB, in order to download data in parts, we can utilize DownloadRangeToStream method. 
3. Amazon S3 also provides us the facility to perform Multipart upload using the APIs exposed through AWS .NET SDK.
1. But first we need to understand the nature and different options the SDK provides and select which best suit of needs.
2. AWS .NET SDK for Multipart upload provides High-Level API and Low-Level API.
3. High-Level API provides us with some sophisticated way of performing few common operations on S3 by encapsulating core level implementations.
4. Low-Level API as you might have guessed, exposes the core Multipart operations on S3. 
5. For our scenario, Low-Level API will be the best fit as we need few detailed control over the operations.
6. Maximum of 10000 parts are allowed per upload.
7. Part size can be of range 5MB to 5GB with an exception to the last part which can be less than 5MB.
4. To handle our scenario, we're going to stick to 100MB as part size in our library.

Library Source code

Following classes has been created to facilitate developers utilize as library encapsulating the implementation logic of the copy operation and provide Request and Response object for ease of use and code management.

1. BlobToS3Manager: Holds the implementation logic of copying BLOB data int S3.
2. IBlobToS3Manager: Prototype of BlobToS3Manager.
3. BlobToS3Request: To hold the BLOB source and S3 target data needed for copy operation.
4. BlobToS3Response: To hold the response of the copy operation.   

You can find the above 4 library files in my github directory here.

Snippet 1 (BlobToS3Manager)

namespace AWSSampleConsoleApp1.BlobToS3
{
    using Amazon.S3.Model;
    using Microsoft.WindowsAzure.Storage.Blob;
    using System;
    using System.Collections.Generic;
    using System.IO;
    using System.Security.Cryptography;
    using System.Threading.Tasks;

    /// <summary>
    /// Handles the BLOB to S3 
    /// </summary>
    public class BlobToS3Manager : IBlobToS3Manager
    {
        /// <summary>
        /// The container not exists.
        /// </summary>
        private const string ContainerNotExists = "BLOB Container doesn't exists.";

        /// <summary>
        /// The BLOB not exists.
        /// </summary>
        private const string BlobNotExists = "BLOB doesn't exists.";

        /// <summary>
        /// Part size to read from BLOB and upload to S3.
        /// </summary>
        private const long PartSize = 104857600; // 100 MB.

        /// <summary>
        /// The BLOB S3 request.
        /// </summary>
        private readonly BlobToS3Request blobToS3Request;
        /// <summary>
        /// Initializes new instance of BLOB to S3 Manager.
        /// </summary>
        /// <param name="blobToS3Request">The BLOB to S3 request.</param>
        public BlobToS3Manager(BlobToS3Request blobToS3Request)
        {
            this.blobToS3Request = blobToS3Request;
        }

        /// <summary>
        /// Copies BLOB to S3.
        /// </summary>
        /// <returns>The BLOB to S3 response.</returns>
        public async Task<BlobToS3Response> CopyFromBlobToS3Async()
        {
            BlobToS3Response blobToS3Response = new BlobToS3Response();
            var validation = await this.Validate();

            if (!validation.Item1)
            {
                return validation.Item2;
            }

            var sourceBlob = validation.Item3;
            await sourceBlob.FetchAttributesAsync();
            var remainingBytes = sourceBlob.Properties.Length;
            long readPosition = 0; // To be used offset / position from where to start reading from BLOB.

            InitiateMultipartUploadRequest initiateMultipartUploadRequest = new InitiateMultipartUploadRequest
            {
                BucketName = this.blobToS3Request.TargetS3Bucket,
                Key = this.blobToS3Request.TargetS3File
            };

            // Will use UploadId from this response.
            InitiateMultipartUploadResponse initiateMultipartUploadResponse = this.blobToS3Request.S3Client.InitiateMultipartUpload(initiateMultipartUploadRequest);
            List<UploadPartResponse> uploadPartResponses = new List<UploadPartResponse>();

            try
            {
                int partCounter = 0; // To increment on each read of parts and use it as part number.
                var sha256 = new SHA256Managed();

                while (remainingBytes > 0)
                {
                    // Determine the size when final block reached as it might be less than Part size. 
                    // Will be PartSize except final block.
                    long bytesToCopy = Math.Min(PartSize, remainingBytes);

                    using (MemoryStream memoryStream = new MemoryStream())
                    {
                        // To download part from BLOB.
                        await sourceBlob.DownloadRangeToStreamAsync(memoryStream, readPosition, bytesToCopy).ConfigureAwait(false);
                        memoryStream.Position = 0;
                        partCounter++;

                        UploadPartRequest uploadRequest = new UploadPartRequest
                        {
                            BucketName = this.blobToS3Request.TargetS3Bucket,
                            Key = this.blobToS3Request.TargetS3File,
                            UploadId = initiateMultipartUploadResponse.UploadId,
                            PartNumber = partCounter,
                            PartSize = bytesToCopy,
                            InputStream = memoryStream
                        };

                        UploadPartResponse uploadPartResponse = this.blobToS3Request.S3Client.UploadPart(uploadRequest);
                        uploadPartResponses.Add(uploadPartResponse);

                        remainingBytes -= bytesToCopy;
                        readPosition += bytesToCopy;

                        // $"Uploaded part with part number {partCounter}, size {bytesToCopy}bytes and remaining {remainingBytes}bytes to read.")

                        // Calculate the checksum value.
                        if (remainingBytes <= 0)
                        {
                            sha256.TransformFinalBlock(memoryStream.ToArray(), 0, (int)bytesToCopy);
                        }
                        else
                        {
                            byte[] bytesToSend = memoryStream.ToArray();
                            sha256.TransformBlock(bytesToSend, 0, (int)bytesToCopy, bytesToSend, 0);
                        }
                    }
                }

                blobToS3Response.Sha256CheckSum = BitConverter.ToString(sha256.Hash).Replace("-", string.Empty);

                CompleteMultipartUploadRequest completeMultipartUploadRequest = new CompleteMultipartUploadRequest
                {
                    BucketName = this.blobToS3Request.TargetS3Bucket,
                    Key = this.blobToS3Request.TargetS3File,
                    UploadId = initiateMultipartUploadResponse.UploadId
                };

                completeMultipartUploadRequest.AddPartETags(uploadPartResponses);

                CompleteMultipartUploadResponse completeMultipartUploadResponse = await this.blobToS3Request.S3Client.CompleteMultipartUploadAsync(completeMultipartUploadRequest).ConfigureAwait(false);

                blobToS3Response.IsSuccess = true;
                blobToS3Response.S3Path = completeMultipartUploadResponse.Location;
            }
            catch (Exception exception)
            {
                blobToS3Response.IsSuccess = false;
                blobToS3Response.Message = exception.Message;

                AbortMultipartUploadRequest abortMultipartUploadRequest = new AbortMultipartUploadRequest
                {
                    BucketName = this.blobToS3Request.TargetS3Bucket,
                    Key = this.blobToS3Request.TargetS3File,
                    UploadId = initiateMultipartUploadResponse.UploadId
                };

                await this.blobToS3Request.S3Client.AbortMultipartUploadAsync(abortMultipartUploadRequest).ConfigureAwait(false);
            }

            return blobToS3Response;
        }

        /// <summary>
        /// Validates the source BLOB is valid.
        /// </summary>
        /// <returns>The resultant tuple.</returns>
        private async Task<Tuple<bool, BlobToS3Response, CloudBlockBlob>> Validate()
        {
            BlobToS3Response blobToS3Response = null;
            CloudBlobContainer cloudBlobContainer = this.blobToS3Request.BlobClient.GetContainerReference(this.blobToS3Request.SourceBlobContainer);

            if (!await cloudBlobContainer.ExistsAsync())
            {
                blobToS3Response = new BlobToS3Response
                {
                    IsSuccess = false,
                    Message = ContainerNotExists
                };

                return new Tuple<bool, BlobToS3Response, CloudBlockBlob>(false, blobToS3Response, null);
            }

            CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(this.blobToS3Request.SourceBlob);

            if (await cloudBlockBlob.ExistsAsync())
            {
                return new Tuple<bool, BlobToS3Response, CloudBlockBlob>(true, null, cloudBlockBlob);
            }

            blobToS3Response = new BlobToS3Response
            {
                IsSuccess = false,
                Message = BlobNotExists
            };

            return new Tuple<bool, BlobToS3Response, CloudBlockBlob>(false, blobToS3Response, null);
        }

    }
}

Snippet 2 (CopyFromBlobToS3: Sample on how to create instances and call the library)

Full source code can be found in my github repo here.

References

• https://docs.microsoft.com/en-us/dotnet/api/microsoft.windowsazure.storage.blob.cloudblob.downloadrangetostream?view=azure-dotnet
• https://docs.microsoft.com/en-us/dotnet/api/microsoft.windowsazure.storage.blob.cloudblob.downloadrangetostreamasync?view=azure-dotnet
• https://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
• https://docs.aws.amazon.com/AmazonS3/latest/dev/qfacts.html

Calculating Hash values / checksum for files while we read them as Streams in chunks

Introduction

You might need to encounter situations where you need to calculate the checksum of file / stream while transmitting across the wire. Nowadays, it is common to transmit the file stream frequently and during that scenario, you need to ensure data has not been corrupted during this transmission process. For which in the receiving end you need to use the same algorithm and recalculate the checksum to ensure the transmitted data is not corrupted.

Scenario

Let me take the same scenario I explained in my previous post Download large files as chunks and upload them into BLOB. In which we downloaded and transmitted large file as stream in chunks. There are lot of articles over web explaining calculating checksum for full file stream. Here we'll see the snippet below for the case of calculating checksum for chunks and get them accumulated at the end.

Tips

HashAlgorithm.TransformBlock and HashAlgorithm.TransformFinalBlock will help you achieve this.

Snippet

public class LargeFileProcessor
    {       
        /// <summary>
        /// Logger instance.
        /// </summary>
        private ILogger logger = new Logger();

        /// <summary>
        /// Download Large File as chunk and upload as chunk into BLOB.
        /// </summary>
        public async Task ProcessLargeFile()
        {
            // Trimmed for brevity.

            string urlToDownload = CloudConfigurationManager.GetSetting("DownloadURL"); // Provide valid URL from where the large file can be downloaded.

            Stopwatch stopwatch = Stopwatch.StartNew();

            try
            {
                using (HttpClient httpClient = new HttpClient())
                {
                    var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, new Uri(urlToDownload))
                    {
                        // To avoid error related to 'An existing connection was forcibly closed by the remote host'. Use Http1.0 instead of Http1.1.
                        Version = HttpVersion.Version10
                    };

                    using (HttpResponseMessage response = await httpClient.SendAsync(httpRequestMessage, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false))
                    {
                        using (Stream stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
                        {
                            const int pageSizeInBytes = 104857600; // 100MB. As Blob chunk max size is 100MB as of now.

                            var sha256 = new SHA256Managed();

                            var bytesRemaing = response.Content.Headers.ContentLength.Value; // Read Total file size from the header.

                            while (bytesRemaing > 0)
                            {
                                var bytesToCopy = (int)Math.Min(bytesRemaing, pageSizeInBytes);
                                var bytesToSend = new byte[bytesToCopy];

                                var bytesCountRead = await ReadStreamAndAccumulate(stream, bytesToSend, bytesToCopy);

                                // Instead of calculating bytes remaining to exit the While loop,  we can use bytesCountRead as bytesCountRead will be 0 when there are no more bytes to read form the stream.   
                                bytesRemaing -= bytesCountRead;

                                // Calculate the checksum value.
                                if (bytesRemaing <= 0)
                                {
                                    sha256.TransformFinalBlock(bytesToSend, 0, bytesCountRead);
                                }
                                else
                                {
                                    sha256.TransformBlock(bytesToSend, 0, bytesCountRead, bytesToSend, 0);
                                }
                            }

                            var checksum = BitConverter.ToString(sha256.Hash).Replace("-", string.Empty);
                            this.logger.WriteLine($"Hash value is : {checksum}");

                            await Task.FromResult(0);
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                this.logger.WriteLine(ex.Message);
                throw;
            }
            finally
            {
                stopwatch.Stop();
                this.logger.WriteLine($"Execution time in mins: {stopwatch.Elapsed.TotalMinutes}");
            }
        }

        /// <summary>
        /// Read the stream and accumulate till it reaches the number of bytes specified to copy.
        /// </summary>
        /// <param name="stream">Stream to be read from.</param>
        /// <param name="bytesToSend">Target byte array that holds the bytes read.</param>
        /// <param name="bytesCountToCopy">The number of bytes to be copied.</param>
        /// <returns>The number of bytes read.</returns>
        private async Task<int> ReadStreamAndAccumulate(Stream stream, byte[] bytesToSend, int bytesCountToCopy)
        {
                        // Trimmed for brevity.
        }

        /// <summary>
        /// Reads the stream with retry when failed. 
        /// </summary>
        /// <param name="stream">Stream to be read from.</param>
        /// <param name="bytesToSend">Target byte array that holds the bytes read.</param>
        /// <param name="bytesCountToCopy">The number of bytes to be copied.</param>
        /// <param name="offset">The byte offset in buffer at which to begin writing data from the stream.</param>
        /// <returns>The number of bytes read.</returns>
        private async Task<int> ReadStreamWithRetry(Stream stream, byte[] bytesToSend, int bytesCountToCopy, int offset)
        {
                        // Trimmed for brevity.
        }
    }

In the above Snippet I've trimmed set of code to just focus on minimalist snippet to portray calculating checksum for chunk read. You can find full source code in my github repo here.

References

• http://peterkellner.net/2010/11/24/efficiently-generating-sha256-checksum-for-files-using-csharp/
• https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.hashalgorithm.transformblock?view=netframework-4.7.1#System_Security_Cryptography_HashAlgorithm_TransformBlock_System_Byte___System_Int32_System_Int32_System_Byte___System_Int32_
• https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.hashalgorithm.transformfinalblock?view=netframework-4.7.1#System_Security_Cryptography_HashAlgorithm_TransformFinalBlock_System_Byte___System_Int32_System_Int32_

Download large files as chunks and upload them into BLOB

Introduction

In real case scenarios there are time where we need to download large files from from a Web resource and save it to Azure BLOB storage. Even though there are few articles over Web which helped, I'm not able to get a end to end working solution for files with size ranging from 20GB to 30GB. Have detailed below challenges and related solutions for your convenience.

Problem statement

Download large files from the Web resource and upload them into Azure BLOB storage.

Challenges and Solutions

1. First we'll hit memory issue when we try to read the full stream and load all bytes into memory. An article here explains well how to avoid this Memory issue.
2. Another article here detailed how to read FileStream  as chunks and upload them into BLOB. Using the details from this and #1 we can try to achieve the solution. 
1. Use Stream.Read by passing respective parameters. In that one of the parameter is maximum number of bytes to read from the current stream. And  the Stream.Read method returns the bytes read. But that also got into issue as the Stream.Read method returns total number of bytes read which can be less than the maximum count parameter. Detailed documentation of Stream.Read can be found here. 
2.  Along with that, we can use CloudBlockBlob.PutBlock method to upload read chunks into BLOB. Points to note here are, each block (chunk) in BLOB can be maximum of 100MB in size and at the max you can have 50000 blocks (100MB x 50000 blocks = 4.75TB). Detailed documentation can be found here. 
3. Even though #2.1 and #2.2 looks straight forward, there might be issues because Read can read less number of bytes which in turn increases the number of blocks (more than 50000 limit) in the Azure BLOB storage.
4. In order to avoid issue mentioned above in #2.3, we need to accumulate the resultant of  Stream.Read till it reaches the expected size (100MB in our case).
3. Even though you fix the issues specified in #1 and #2 above, you might face exception stating An existing connection was forcibly closed by the remote host. . Though I'm not able to identify the solution for this, have found workaround using the articles here and here. Use Http1.0 instead of Http1.1.

Code samples can be found in my github repo here.

Snippet 

public class LargeFileProcessor
    {
        /// <summary>
        /// Logger instance.
        /// </summary>
        private ILogger logger = new Logger();

        /// <summary>
        /// Retry count.
        /// </summary>
        private int retryCount = 5;

        /// <summary>
        /// Time delay for retry.
        /// </summary>
        private TimeSpan delay = TimeSpan.FromSeconds(10);

        /// <summary>
        /// Download Large File as chunk and upload as chunk into BLOB.
        /// </summary>
        public async Task ProcessLargeFile()
        {
            // Create Storage account reference.
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageAccount"));

            // Create the blob client.
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

            // Retrieve reference to a container.
            CloudBlobContainer container = blobClient.GetContainerReference(CloudConfigurationManager.GetSetting("ContainerName"));
            container.CreateIfNotExists();

            // Create Blob reference.
            CloudBlockBlob blob = container.GetBlockBlobReference(CloudConfigurationManager.GetSetting("BlobFileName"));

            string urlToDownload = CloudConfigurationManager.GetSetting("DownloadURL"); // Provide valid URL from where the large file can be downloaded.

            Stopwatch stopwatch = Stopwatch.StartNew();

            try
            {
                using (HttpClient httpClient = new HttpClient())
                {
                    var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, new Uri(urlToDownload))
                    {
                        // To avoid error related to 'An existing connection was forcibly closed by the remote host'. Use Http1.0 instead of Http1.1.
                        Version = HttpVersion.Version10
                    };

                    using (HttpResponseMessage response = await httpClient.SendAsync(httpRequestMessage, HttpCompletionOption.ResponseHeadersRead).ConfigureAwait(false))
                    {
                        using (Stream stream = await response.Content.ReadAsStreamAsync().ConfigureAwait(false))
                        {
                            const int pageSizeInBytes = 104857600; // 100MB. As Blob chunk max size is 100MB as of now.

                            var blockIds = new List<string>();
                            var sha256 = new SHA256Managed();

                            var bytesRemaing = response.Content.Headers.ContentLength.Value; // Read Total file size from the header.
                            int blockIdentifier = 0;

                            while (bytesRemaing > 0)
                            {
                                blockIdentifier++;
                                var bytesToCopy = (int)Math.Min(bytesRemaing, pageSizeInBytes);
                                var bytesToSend = new byte[bytesToCopy];

                                var bytesCountRead = await ReadStreamAndAccumulate(stream, bytesToSend, bytesToCopy);

                                // Instead of calculating bytes remaining to exit the While loop,  we can use bytesCountRead as bytesCountRead will be 0 when there are no more bytes to read form the stream.   
                                bytesRemaing -= bytesCountRead;

                                this.logger.WriteLine($"bytes read: {bytesCountRead}");
                                this.logger.WriteLine($"bytes remaining: {bytesRemaing}");

                                string base64BlockId = Convert.ToBase64String(ASCIIEncoding.ASCII.GetBytes(string.Format("largefile1BlockId{0}", blockIdentifier.ToString("0000000"))));
                                blockIds.Add(base64BlockId);

                                // Calculate the checksum value.
                                if (bytesRemaing <= 0)
                                {
                                    sha256.TransformFinalBlock(bytesToSend, 0, bytesCountRead);
                                }
                                else
                                {
                                    sha256.TransformBlock(bytesToSend, 0, bytesCountRead, bytesToSend, 0);
                                }

                                await blob.PutBlockAsync(base64BlockId, new MemoryStream(bytesToSend), null);
                            }

                            var checksum = BitConverter.ToString(sha256.Hash).Replace("-", string.Empty);
                            this.logger.WriteLine($"Hash value is : {checksum}");
                            await blob.PutBlockListAsync(blockIds);

                            await Task.FromResult(0);
                        }
                    }
                }
            }
            catch (Exception ex)
            {
                this.logger.WriteLine(ex.Message);
                throw;
            }
            finally
            {
                stopwatch.Stop();
                this.logger.WriteLine($"Execution time in mins: {stopwatch.Elapsed.TotalMinutes}");
            }
        }

        /// <summary>
        /// Read the stream and accumulate till it reaches the number of bytes specified to copy.
        /// </summary>
        /// <param name="stream">Stream to be read from.</param>
        /// <param name="bytesToSend">Target byte array that holds the bytes read.</param>
        /// <param name="bytesCountToCopy">The number of bytes to be copied.</param>
        /// <returns>The number of bytes read.</returns>
        private async Task<int> ReadStreamAndAccumulate(Stream stream, byte[] bytesToSend, int bytesCountToCopy)
        {
            int bytesReadSoFar = 0;

            while (bytesReadSoFar < bytesCountToCopy)
            {
                var currentBytesCountRead = await ReadStreamWithRetry(stream, bytesToSend, bytesCountToCopy - bytesReadSoFar, bytesReadSoFar).ConfigureAwait(false);
                bytesReadSoFar += currentBytesCountRead;
            }

            return bytesReadSoFar;
        }

        /// <summary>
        /// Reads the stream with retry when failed. 
        /// </summary>
        /// <param name="stream">Stream to be read from.</param>
        /// <param name="bytesToSend">Target byte array that holds the bytes read.</param>
        /// <param name="bytesCountToCopy">The number of bytes to be copied.</param>
        /// <param name="offset">The byte offset in buffer at which to begin writing data from the stream.</param>
        /// <returns>The number of bytes read.</returns>
        private async Task<int> ReadStreamWithRetry(Stream stream, byte[] bytesToSend, int bytesCountToCopy, int offset)
        {
            int currentRetry = 0;
            for (; ; )
            {
                try
                {
                    var bytesRead = await stream.ReadAsync(bytesToSend, offset, bytesCountToCopy);
                    return bytesRead;
                }
                catch (Exception ex)
                {
                    this.logger.WriteLine($"Operation Exception : {ex.Message}");

                    currentRetry++;

                    // Check if it is within the retry count specified.
                    if (currentRetry > this.retryCount)
                    {
                        // Rethrow the exception if it more than the retry attempt.
                        throw;
                    }
                }

                // Wait to retry the operation.
                await Task.Delay(delay);
            }

        }
    }

Full source code has been provided here.

References

• http://www.tugberkugurlu.com/archive/efficiently-streaming-large-http-responses-with-httpclient
• https://www.red-gate.com/simple-talk/cloud/platform-as-a-service/azure-blob-storage-part-4-uploading-large-blobs/
• https://weblog.west-wind.com/posts/2014/jan/29/using-net-httpclient-to-capture-partial-responses
• https://inside.covve.com/uploading-block-blobs-larger-than-256-mb-in-azure/
• http://www.alexandre-gomes.com/?p=144
• https://docs.microsoft.com/en-us/rest/api/storageservices/put-block-list#remarks
• https://docs.microsoft.com/en-us/dotnet/api/system.io.stream.read?view=netframework-4.7.1
• https://tutel.me/c/programming/questions/33233780/systemnethttphttprequestexception+error+while+copying+content+to+a+stream
• https://social.msdn.microsoft.com/Forums/en-US/c620ce2c-c512-4c9f-a481-521ecd260039/systemioioexception-unable-to-read-data-from-the-transport-connection-the-connection-was-closed?forum=vstswebtest

 

Time triggered / Scheduler service with Azure Service Fabric

Introduction

Azure Service Fabric Services can be configured to exhibit trigger based behavior similar to WebJobs and Azure Functions.

Time triggered / Scheduler Service in Azure Service Fabric

To create a time triggered (scheduler) Service in Azure Service Fabric, I can think of following two quick options among the possible ways.

1. Create a time triggered WebJob and add/deploy it as a guest executable in Azure Service Fabric.
2. Create Azure Service Fabric Stateless Service and implement the listener to handle jobs.

Here I'm taking the second approach which involves the respective listener to be created. Rather than writing new Scheduling framework, I'm using the Quartz.Net framework. I'm using the CRON expression behavior to have the behavior inline with the time trigger behavior of WebJobs and Azure Functions.

I'll be explaining two approaches below.

1. Simple Time trigger Service
2. Time trigger Service using Dependency Injection

Simple Time trigger Service

• Uses Quartz.Net framework.
• Without Dependency Injection.
• Listerner gets all the IJob types from the current assembly with JobInfoAttribute decorated.
• JobInfoAttribute holds the name and CRON expression of the job.
• Code samples can be found in my github repo here.

Snippet 1 (SimpleTimeTriggerListener)

using Microsoft.ServiceFabric.Services.Communication.Runtime;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Reflection;
using Quartz;
using TimeTriggerService;
using Quartz.Impl;

namespace SimpleTimeTriggerService
{
    public class SimpleTimeTriggerListener : ICommunicationListener
    {
        private IScheduler scheduler;

        public void Abort()
        {
            scheduler.Shutdown(false);
        }

        public Task CloseAsync(CancellationToken cancellationToken)
        {
            scheduler.Shutdown(true);
            return Task.FromResult(true);
        }

        public Task<string> OpenAsync(CancellationToken cancellationToken)
        {
            var tasks =
                Assembly.GetExecutingAssembly()
                    .GetTypes()
                    .Where(
                        t =>
                            typeof(IJob).IsAssignableFrom(t) &&
                            Attribute.IsDefined(t, typeof(JobInfoAttribute)));

            foreach (var task in tasks)
            {
                var schedulerFactory = new StdSchedulerFactory();
                scheduler = schedulerFactory.GetScheduler();
                var jobInfo = Attribute.GetCustomAttribute(task, typeof(JobInfoAttribute)) as JobInfoAttribute;
                var jobName = jobInfo == null ? task.Name : jobInfo.Name;
                var job = new JobDetailImpl(jobName, null, task);

                var trigger =
                    TriggerBuilder.Create()
                        .WithIdentity($"{jobName}Trigger", null)
                        .WithCronSchedule(jobInfo.CronExpression)
                        .ForJob(job)
                        .Build();
                scheduler.ScheduleJob(job, trigger);                
            }

            scheduler.Start();

            return Task.FromResult("Sample Simple Job scheduler");
        }
    }
}

Time trigger Service using Dependency Injection

• Uses Quartz.Net framework.

• With Dependency Injection using Autofac (Autofac integration package for Quartz.Net).

• Listerner gets all the IJob types from the current assembly with JobInfoAttribute decorated.

• JobInfoAttribute holds the name and CRON expression of the job.

• Code samples can be found in my github repo here.

Snippet 2 (TimeTriggerListener with Autofac)

using Microsoft.ServiceFabric.Services.Communication.Runtime;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Threading;
using Autofac;
using Quartz;
using Autofac.Extras.Quartz;
using System.Reflection;

namespace TimeTriggerService
{
    internal class TimeTriggerListener : ICommunicationListener
    {
        private IScheduler scheduler;

        public void Abort()
        {
            scheduler.Shutdown(false);
        }

        public Task CloseAsync(CancellationToken cancellationToken)
        {
            scheduler.Shutdown(true);
            return Task.FromResult(true);
        }

        public Task<string> OpenAsync(CancellationToken cancellationToken)
        {
            ContainerBuilder builder = new ContainerBuilder();
            builder.RegisterModule(new QuartzAutofacFactoryModule());
            builder.RegisterModule(new QuartzAutofacJobsModule(Assembly.GetExecutingAssembly()));
            builder.RegisterAssemblyTypes(Assembly.GetExecutingAssembly()).Where(x => typeof(IJob).IsAssignableFrom(x)).As<IJob>();
            builder.RegisterType<Logger>().As<ILogger>();
            builder.RegisterType<JobScheduler>().AsSelf();
            IContainer container = builder.Build();
            ConfigureScheduler(container);

            return Task.FromResult("Sample Job scheduler");
        }
        
        private void ConfigureScheduler(IContainer container)
        {
            IEnumerable<IJob> jobList = container.Resolve<IEnumerable<IJob>>();
            var jobScheduler = container.Resolve<JobScheduler>();
            this.scheduler = jobScheduler.Start();
        }
    }
}

Snippet 3 (JobScheduler)

using Quartz;
using Quartz.Impl;
using System;
using System.Collections.Generic;

namespace TimeTriggerService
{
    public class JobScheduler
    {
        private IScheduler scheduler;
        private IEnumerable<IJob> jobs;
        public JobScheduler(IScheduler schedulerParam, IEnumerable<IJob> jobsParam)
        {
            this.scheduler = schedulerParam;
            this.jobs = jobsParam;
        }

        public IScheduler Start()
        {
            foreach (var job in jobs)
            {
                var task = job.GetType();
                var jobInfo = Attribute.GetCustomAttribute(task, typeof(JobInfoAttribute)) as JobInfoAttribute;
                var jobName = jobInfo == null ? task.Name : jobInfo.Name;
                var jobDetail = new JobDetailImpl(jobName, null, task);
                var trigger =
                    TriggerBuilder.Create()
                        .WithIdentity($"{jobName}Trigger", null)
                        .WithCronSchedule(jobInfo.CronExpression)
                        .ForJob(jobDetail)
                        .Build();
                scheduler.ScheduleJob(jobDetail, trigger);
            }
            scheduler.Start();

            return scheduler;
        }
    }
}

Full source code has been provided here.

Applying strict boolean behavior in ASP.NET WEB API JSON request model

Introduction

In ASP.NET WEB API / MVC in the request input model if you have some property that accepts bool type with content type application/json, it might even accept numbers which will be converted to bool type. If you send numbers as input for bool, it'll be converted to false for 0 and true for everything else (with an exception observed - 08 and 09 as null). 

Problem statement

To restrict from numbers being accepted as input for bool type property in JSON content type.

Solution

1. Create a converter that inherits from Newtonsoft.Json.JsonConverter.
2. Override ReadJson method, check the ValueType for bool type and throw JSON serializer exception if not.

Snippet (Converter)

public class BooleanConverter : JsonConverter
{
    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.ValueType != typeof(bool))
        {
            throw new JsonSerializationException("Invalid data type");
        }
        return reader.Value;
    }

    public override bool CanConvert(Type objectType)
    {
        return objectType == typeof(bool);
    }
}

Calculate execution time for your ASP.NET WEB API controller actions using Autofac action filter

Requirement

• Calculate execution time of ASP.NET WEB API actions
• Create a common code that can calculate and log the details
• Project uses Autofac for Dependency Injection.

If you've integrated Azure Application Insights to your WEB API, automatically it'll log/capture the execution time of each request. If you need it as additional logging or in place where Application Insights not associated, this will be helpful. Let us get into the code directly.

Approach 

1. We will be using ActionFilter to write the logic for calculating the execution time.
2. In case of WEB API that uses Autofac as dependency resolver, we can utilize the IAutofacActionFilter to achieve the same.
3. Register the respective action filter globally to execute for all controllers and actions, so that you don't need to worry when you add new controllers/actions.
4. In the below example I provided the sample snippet related to Autofac based solution. Ignore the Custom logger in the below example as it is just for portraying the sample.

Snippet1: (Action Filter that calculates the execution time and logs)

using System;

using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Web;
using System.Web.Http.Controllers;
using System.Web.Http.Filters;
using Autofac.Integration.WebApi;

namespace TestWebAPI1.Utils
{
    public class ExecutionTimeActionFilterAttribute : IAutofacActionFilter
    {
        private const string StopwatchInstanceKey = "StopwatchInstance";

        private readonly ICustomLogger1 logger;

        public ExecutionTimeActionFilterAttribute(ICustomLogger1 loggerParam)
        {
            this.logger = loggerParam;
        }


        public void OnActionExecuting(HttpActionContext actionContext)
        {
            Stopwatch stopwatchInstance = new Stopwatch();
            actionContext.Request.Properties[StopwatchInstanceKey] = stopwatchInstance;
            stopwatchInstance.Start();
        }

        public void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
        {
            if (!actionExecutedContext.Request.Properties.ContainsKey(StopwatchInstanceKey))
            {
                return;
            }

            Stopwatch stopwatchInstance = actionExecutedContext.Request.Properties[StopwatchInstanceKey] as Stopwatch;
            if (stopwatchInstance == null)
            {
                return;
            }

            stopwatchInstance.Stop();
            string actionName = actionExecutedContext.ActionContext.ActionDescriptor.ActionName;

            this.logger.WriteLine($"Execution time of action : {actionName} in controller : {actionExecutedContext.ActionContext.ControllerContext.ControllerDescriptor.ControllerName} is {stopwatchInstance.ElapsedMilliseconds} milliseconds");
        }
    }
}

Snippet 2: (Autofac registration in WebApiConfig)

namespace TestWebAPI1
{
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            // Web API configuration and services
            // Configure Web API to use only bearer token authentication.
            config.SuppressDefaultHostAuthentication();
            config.Filters.Add(new HostAuthenticationFilter(OAuthDefaults.AuthenticationType));

            // Web API routes
            config.MapHttpAttributeRoutes();

            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );

            var builder = new ContainerBuilder();
            
            builder.RegisterType<CustomLogger1>().As<ICustomLogger1>().InstancePerDependency();
            
            builder.RegisterApiControllers(Assembly.GetExecutingAssembly());
            
            builder.RegisterWebApiFilterProvider(config);
            builder.Register(c => new ExecutionTimeActionFilterAttribute(c.Resolve<ICustomLogger1>()))
                .AsWebApiActionFilterFor<ApiController>()
                .InstancePerRequest();

            var container = builder.Build();
            config.DependencyResolver = new AutofacWebApiDependencyResolver(container);
        }
    }
}

References

The concept has been taken based on the suggestions by some blogs and stackoverflow forums (unable to recall exact article).

Web Deployment package Zip not getting created even after specifying required MSBuild args in Visual Studio Build task

Introduction

Visual Studio Team Services (VSTS) provide the facilitation to incorporate Continuous Integration and Continuous deployment. For Azure WebApp/WebJobs Documentation and Blogs are available (like this, this ) to guide you how to achieve those. In this post we're going to have a look at scenario where the Web deployment package itself might not get created and solution for that.

Scenario

• Visual Studio build task has been added and configured appropriately.
• Respective MSBuild arguments "/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=$(Build.BinariesDirectory)" has been provided.

But even after that Web deployment package zip file not getting created.

Possible Cause and Solution

Ensure you have the Microsoft.Web.WebJobs.Publish NuGet package added to the respective project. If the package is not associated, in your CI execution Buildstep won't throw any exception, rather it simply won't create the web deployment zip package.

Visual Studio 2017 - Azure Functions template unavailable

After installing Visual Studio 2017 version 15.3 by including Azure development workload, sometimes Azure Functions template might not be available for you in the Add New Project dialog. 

Solution
In such case, go to Tools >> Extensions and Updates... , verify update for Azure Functions and Web Jobs Tools and install the update.

Exception when SignalR hosted using OWIN by Azure worker

I came across the following error when I tried to host SignalR using OWIN by Azure worker role.

Could not load file or assembly 'Microsoft.Owin, Version=2.0.2.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. 

Upon searching over the web, I came across few links (specified some in References).

Reason is specified over here.

Fix

We need to add few configuration (specified in Snippet 1) in app.config file of the project where we have specified the SignalR. mapping code.

While incorporating the fix, we need to observe it and tweak the new version accordingly, if required. Just verify the packages.config to understand which version of Microsoft.Owin and Microsoft.Owin.Security dll got installed. Most of the time, Microsoft.Owin dll version might have got changed. Ensure the new version specified in the app.config file to be matched with the one you've installed / specified in packages.config.

Snippet 1: (app.config)

<runtime>
  <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
    <dependentAssembly>
      <assemblyIdentity name="Microsoft.Owin" publicKeyToken="31bf3856ad364e35" culture="neutral" />
      <bindingRedirect oldVersion="0.0.0.0-2.1.0.0" newVersion="2.1.0.0" />
    </dependentAssembly>
    <dependentAssembly>
      <assemblyIdentity name="Microsoft.Owin.Security" publicKeyToken="31bf3856ad364e35" culture="neutral" />
      <bindingRedirect oldVersion="0.0.0.0-2.0.2.0" newVersion="2.0.2.0" />
    </dependentAssembly>
  </assemblyBinding>
</runtime>

Snippet 2: (packages.config)

<package id="Microsoft.Owin" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Cors" version="2.1.0" targetFramework="net45" />
  <package id="Microsoft.Owin.Diagnostics" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Host.HttpListener" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Hosting" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.SelfHost" version="2.0.2" targetFramework="net45" />

References:

http://stackoverflow.com/questions/20083185/fileloadexception-when-hosting-signalr-at-azure-worker-role-with-f
https://nuget.codeplex.com/workitem/3827

Updating 'Microsoft.Data.OData 5.2.0' to 'Microsoft.Data.OData 5.3.0' failed

Updating 'Microsoft.Data.OData 5.2.0' to 'Microsoft.Data.OData 5.3.0' failed. Unable to find a version of 'Microsoft.AspNet.WebApi.OData' that is compatible with 'Microsoft.Data.OData 5.3.0'.

WCF Data Services 5.3.0 RTW has been released few days back. Once you install it using the tools installer, it will update the "Add Service Reference" tooling integration in Visual Studio, after which it'll create the code based on the WCF Data Services 5.3.0.  In such case, for now if you try to add service reference to a WCF Data Service in an ASP.NET MVC 4 project you might end up getting the error specified at the top of this page.

As mentioned in the error, it seems that the respective "Microsoft.AspNet.WebApi.OData" dll is not available in the last ASP.NET MVC 4 update. Hope this will be made available in the next related update.

Solution: Uninstall the respective program installed using the WCF Data Services 5.3.0 RTW tools installer. If you try the service reference now it'll get you with Microsoft.Data.OData 5.2.0 in the client. This way we can go with WCF Data Service created with 5.3.0 version and the respective client libraries (proxy....) with 5.2.0 (in ASP.NET MVC 4 client projects). Just tried locally, but didn't verified entire features.

As an alternate try with Nuget packages.

WCF Data Services - OData - Initial errors developers encounters

When authoring WCF Data Services - OData, lot of developers tries to view it in the browser during several stages of developement. Mostly they will come across the following error at initial stages.

The server encountered an error processing the request. See  server logs for more details.

When we think of creating WCF Data services, its better to have the following minimal understanding.

WCF Data services / OData service : A .Net framework component which is used to create services that uses Open Data Protocol (OData). This service exposes data through the endpoint which allows to access the data.

OData client libraries : Though not strictly required, several client libraries available for easy and effective access of OData format. Based on the client applications we create, we can make use of the respective client libraries. For example we have separate libraries for .Net, java, javascript, Silverlight, etc.

OData data model : Makes use of the exiting .Net member called Entity Data Model(EDM).  The data to be represented need to be modeled using EDM.

OData Protocol : This protocol is REST based. Defines  how the data model will be represented over the wire - either in XML based format defined by ATOM or in JSON format.

Now, let us look at the cause for the issue.

In the InitializeService method of the svc.cs file, do ensure whether the SetEntitySetAccessRule is specified for the respective entity and most important, the name of the entity set should be pluralized (if opted while creating the entity data model).

Snippet 1

// This method is called only once to initialize service-wide policies.
public static void InitializeService(DataServiceConfiguration config)
{
    // TODO: set rules to indicate which entity sets and service operations are visible, updatable, etc.
    // Examples:
    config.SetEntitySetAccessRule("Employees", EntitySetRights.AllRead);
    config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
}

References

http://msdn.microsoft.com/en-us/data/hh237663.aspx (By David Chappell)

http://msdn.microsoft.com/en-us/library/ff478141.aspx

OData : Cross domain OData request using jQuery, JSONP

I've been doing a POC where I had a situation to make a request to a OData Service (service which exposed the data as OData  - Open Data Protocol) using jQuery. During my initial attempt, I got an error from the web browser which is due to the same origin policy of the web browser.  Usually Client side (request from browser using javascript) cannot be allowed to make a request to a resource that exists in another domain of which HTML <script> element is an exception. In such cases JSONP (JSON with padding) and CORS (Cross Origin Resource Sharing) is made use. Among the two, JSONP requests are commonly used right now and CORS is expected to catch the mass. In this post we'll focus on the JSONP model request for OData Service.

To get a quick idea on the need for JSONP and how that stuff works, please have a quick glimpse here.  The code snippets I represent in this post will use the Nerd Dinner OData Feed for easier understanding.

After deciding to make the JSONP request, I used jQuery.ajax() method with url : http://www.nerddinner.com/Services/OData.svc/Dinners?$format=json & jsonpCallback option which resulted in error. Upon analyzing it, found out the error is "Uncaught SyntaxError: Unexpected token : " and the error is because the returned data is a simple JSON format and not wrapped with the callback method which is expected. Then I searched and found out the right format to make the OData JSONP request. The option is available as a OData query $callback.

Following are the three different alterations we can do to make a cross domain OData request using jQuery and JSONP.

Snippet 1:

$(document).ready(function () {

    $.ajax({
        url: 'http://www.nerddinner.com/Services/OData.svc/Dinners?$format=json&$callback=nerdCallback',
        dataType: "jsonp"
    });
});

function nerdCallback(result) {
    //Manipulate the resultant here....
    alert("Result count : " + $(result["d"]["results"]).length);
}

Snippet 2:

$(document).ready(function () {

    $.ajax({
        url: 'http://www.nerddinner.com/Services/OData.svc/Dinners?$format=json&$callback=?',
        dataType: "jsonp",
        jsonpCallback: "nerdCallback"
    });
});

function nerdCallback(result) {
    //Manipulate the resultant here....
    alert("Result count : " + $(result["d"]["results"]).length);
}

Snippet 3:

$(document).ready(function () {

    $.ajax({
        url: 'http://www.nerddinner.com/Services/OData.svc/Dinners?$format=json&$callback=?',
        dataType: "jsonp",
        success: function (result) {
            //Manipulate the resultant here....
            alert("Result count : " + $(result["d"]["results"]).length);
        }
    });
});

References:
http://en.wikipedia.org/wiki/JSONP
http://msdn.microsoft.com/en-us/library/ee834511.aspx
http://www.odata.org/

ASP.NET MVC 3 - Effectively using Model metadata

In ASP.NET MVC 3, most developers might have observed the existence of the HTML helpers - DisplayForModel and EditorForModel. This post is just to give a hint on how effectively we can use the specified HTML helpers and hence we won't have any code Snippets here.

I frequently come across situations of creating proof of concept and several sample applications to evaluate certain implementations / metrics. In such cases, lot of time I really worried on creating a sample data entry / display page which consumes my time, because my core objective is to evaluate my target as early as possible. The EditorForModel HTML helper in ASP.NET MVC actually saves me lot of time and help me to just focus on my target while doing such POCs.

While doing so I thought some simple tweaks, like a way to represent display name, order ... for the property might help me to tune my sample page further. To achieve that, I got the help of DataAnnotations / Metadata.

Some sample requirements are

• Need to hide a column - ScaffoldColumn(false)
• Hidden value - HiddenInput
• Display a different name - Display(Name = "Employee Name")
• Display in a custom order - Display(Order=1)
• Display format with different type than default - DataType(DataType.MultilineText)

Using these Metadata, we can, hide a property from creating HTML elements for that, create a hidden value element for a property, display a different name than that of the property, display in a custom order we need, specify what type of HTML element and what format can be used, etc.

Once you start explore other parameters of attributes and options, you can achieve lot of cases with minimal effort. Really its a time saving mechanism. If none of the options fits our scenario, then we can do the regular way.   

ASP.NET MVC 3 - Using Ninject for Dependency Injection

This post is to show how to implement Dependency Injection in ASP.NET MVC 3 application with simple steps. For Details on Dependency Injection (DI) please refer to my article here.

For the sake of simplicity and to understand the idea, I have simply specified all the classes in same project(web application here). In real time scenario this might extend to multiple class libraries based on the nature of the project.

In our example, we're targeting a Mark sheet creation with minimal and predefined data. MarkSheetController expects a MarksheetModel and the MarkSheetModel expects a MarkEvaluator which we're going to achieve with Constructor injection as follows.

Implementaiton Details

First we need to install the Ninject.MVC3 Nuget package using Nuget manager. For details on Nuget manager please verify my post here. Upon installing the package, a file called NinjectWebCommon.cs will get created under the folder called App_Start.  In the  static method called RegisterServices, add the statement to register the dependencies with Ninject container. Following Snippets are self explanatory to understand the concept.

Snippet 1: (App_Start/NinjectWebCommon.cs => NinjectWebCommon.RegisterServices())

private static void RegisterServices(IKernel kernel)
{
    kernel.Bind<IMarkSheetModel>().To<MarkSheetModel>();
    kernel.Bind<IMarkEvaluator>().To<MarkEvaluator>();
}

In the above snippet I've registered with Ninject that MarkSheetModel concrete class is bound to IMarkSheetModel and MarkEvaluator concrete class is bound to IMarkEvaluator interface. Based on this Ninject resolves the dependency by creating instance for the respective concrete class and pass it in the place of the related interface specification. Following snippet shows how we're making use of constructor injection to achieve that.

Snippet 2: (MarkSheetController - note the IMarkSheetModel parameter in the constructor)

public class MarkSheetController : Controller
{
    private IMarkSheetModel model;


    public MarkSheetController(IMarkSheetModel modelParam)
    {
        model = modelParam;
    }

    //
    // GET: /MarkSheet/

    public ActionResult Index()
    {
        model.LoadDetails();
        return View(model);
    }

}

Snippet 3: (MarkSheetController - note the IMarkEvaluator parameter in the constructor)

public class MarkSheetModel : IMarkSheetModel
{
    private IMarkEvaluator evaluator;

    public string StudentId { get; set; }
    public string StudentName { get; set; }
    public IEnumerable<Subject> SubjectList { get; set; }
    public int Total { get; private set; }
    public decimal Average { get; private set; }
    public string Grade { get; private set; }
    
    public MarkSheetModel(IMarkEvaluator evaluatorParam)
    {
        evaluator = evaluatorParam;
    }

    public void LoadDetails()
    {
        this.StudentId = "S1001";
        this.StudentName = "Sample";

        this.SubjectList = new List<Subject> { new Subject{ SubjectId="Sub0001", SubjectName ="Sub 1", Score = 78},
        new Subject{ SubjectId="Sub0002", SubjectName ="Sub 2", Score = 84},
        new Subject{ SubjectId="Sub0003", SubjectName ="Sub 3", Score = 72},
        new Subject{ SubjectId="Sub0004", SubjectName ="Sub 4", Score = 69}};

        this.Total = this.evaluator.CalculateTotal(this.SubjectList);
        this.Average = this.evaluator.CalculateAverage(this.SubjectList);
        this.Grade = this.evaluator.CalculateGrade(this.SubjectList);
    }

}

public interface IMarkSheetModel
{
    void LoadDetails();
    IEnumerable<Subject> SubjectList { get; set; }
    string StudentId { get; set; }
    string StudentName { get; set; }
}

public class Subject
{
    public string SubjectId { get; set; }
    public string SubjectName { get; set; }
    public int Score { get; set; }
}

Snippet 4:

public class MarkEvaluator : IMarkEvaluator
{
    public int CalculateTotal(IEnumerable<Subject> subjectList)
    {
        return subjectList.Sum(su => su.Score);
    }

    public decimal CalculateAverage(IEnumerable<Subject> subjectList)
    {
        return Convert.ToDecimal(CalculateTotal(subjectList)) / subjectList.Count();
    }

    public String CalculateGrade(IEnumerable<Subject> subjectList)
    {
        decimal averageScore = CalculateAverage(subjectList);

        if (averageScore > 80)
        {
            return "Grade A";
        }
        else if (averageScore > 70)
        {
            return "Grade B";
        }
        else if (averageScore > 60)
        {
            return "Grade C";
        }
        else
        {
            return "Grade D";
        }
    }
}

public interface IMarkEvaluator
{
    decimal CalculateAverage(IEnumerable<Subject> subjectList);
    string CalculateGrade(IEnumerable<Subject> subjectList);
    int CalculateTotal(IEnumerable<Subject> subjectList);
}

While executing the above snippets, the constructor of the MarkSheetController and MarkSheetModel is supplied with the respective instance.

References:
https://github.com/ninject/ninject.web.mvc/wiki/MVC3

ASP.NET MVC 3 - Converting / Serializing .Net objects into JSON format in View

In ASP.NET MVC often we might need to use / manipulate Model objects or its property inside the javascript. In order to achieve that, we usually try to serialize the .Net object and store it in a javascript variable which will then be accessed further in javascript.

Using Razor view engine, we can achieve this easily. In the following, I'll explain two approaches with a simple View(.cshtml) snippet to understand the concept. 

Approach 1: (Using Json.Encode(...))

Snippet 1: (Index.cshtml)

@model Web.POC.Models.FeedModel
@{
    ViewBag.Title = "Feed viewer";
}

<script type="text/javascript">
    var entries = @Html.Raw(Json.Encode(Model.FeedEntries));    
</script>

Approach 2: (Using JavaScriptSerializer)

Snippet 2: (Index.cshtml)

@model Web.POC.Models.FeedModel
@{
    ViewBag.Title = "Feed viewer";
}

@{
   System.Web.Script.Serialization.JavaScriptSerializer serializer = new System.Web.Script.Serialization.JavaScriptSerializer();
}
<script type="text/javascript">
    var entries = @Html.Raw(serializer.Serialize(Model.FeedEntries));    
</script>

Assumptions:

As specified, the above snippets are just to understand the concept. In that, consider that we're having a model "FeedModel" which holds a property called "FeedEntries" of some custom type which we're trying to convert into JSON object.

Also note that we need to use the HTML.Raw() to indicate that the output should not be HTML encoded.

ASP.NET - Avoid multiple postbacks for control set using validators by disabling the button

In ASP.NET disabling the button when a postback got initiated through button click and avoiding multiple postbacks or submits until the specific response is processed is one of the common thing that we need to implement. This is especially  needed for some long running operations. During my initial days, I achieve this as specified here. Later, I followed as specified here in an article by Abhijit Jana. Few days back one of my friend asked me about how to disbale the button to avoid multiple submits / postbacks for control set that are using ASP.NET validation. To achieve that, I tried a sample and came up with two approaches.

Let us consider a scenario that we have a text box, a required field validator and button tied together using validation group.

Approach 1: (Using jQuery and a button proxy)

• Apply CSS style="display:none;"  to the button you want to disable.
• Include a new button for display purpose and add a jQuery that validates the validation group using the function Page_ClientValidate() passing the validationgroup as parameter. 
• Based on the return value of Page_ClientValidate(), raise the click event of the hidden button using jQuery and disable the display button. 

The following script snippet helps us to achieve that.

Snippet 1:(script)

<script type="text/javascript">
    $(document).ready(function () {
        initAction();
    });

    function initAction() {
        $('#Button2').click(function (event) {
            event.preventDefault();

            if (Page_ClientValidate('ValGroup1')) {
                this.disabled = true;
                $('#Button1').click();
            }

        });
    }
</script>

Here is the full implementation of the same.

Snippet 2:(DisableButtonOnClickApp1.aspx)

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="DisableButtonOnClickApp1.aspx.cs"
    Inherits="DisableButtonOnClickApp1" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
        <script type="text/javascript" src="http://code.jquery.com/jquery-1.7.1.min.js"></script>
    <script type="text/javascript">
        $(document).ready(function () {
            initAction();
        });

        function initAction() {
            $('#Button2').click(function (event) {
                event.preventDefault();

                if (Page_ClientValidate('ValGroup1')) {
                    this.disabled = true;
                    $('#Button1').click();
                }

            });
        }
    </script>

</head>
<body>
    <form id="form1" runat="server">
    <div>
        <table border="0" cellpadding="0" cellspacing="0">
            <tr>
                <td>
                    <asp:Label ID="Label1" runat="server" Text="Label 1 : "></asp:Label>
                </td>
                <td>
                    <asp:TextBox ID="TextBox1" runat="server" ValidationGroup="ValGroup1"></asp:TextBox>
                    <asp:RequiredFieldValidator ID="RequiredFieldValidator1" runat="server" ErrorMessage="RequiredFieldValidator"
                        ControlToValidate="TextBox1" ValidationGroup="ValGroup1"></asp:RequiredFieldValidator>
                </td>
            </tr>
            <tr>
                <td colspan="2" style="text-align: center;">
                    <asp:Button ID="Button1" runat="server" Text="Hide Button" Style="display: none;"
                        ValidationGroup="ValGroup1" onclick="Button1_Click" />
                    <asp:Button ID="Button2" runat="server" Text="Display Button" />
                </td>
            </tr>
            <tr>
            <td colspan="2" style="text-align: center;">
                <asp:Label ID="StatusMessageLabel" runat="server" Text=""></asp:Label>
            </td>
            </tr>
        </table>
    </div>
    </form>
</body>
</html>

Snippet 3:(DisableButtonOnClickApp1.aspx.cs)

public partial class DisableButtonOnClickApp1 : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {

    }

    protected void Button1_Click(object sender, EventArgs e)
    {
        StatusMessageLabel.Text = "Button 1 click .......";
    }
}

If you are not much interested in using one additional control to achieve this, the following approach might help you.

Approach 2: (Using GetPostBackEventReference with PostBackOptions and UseSubmitBehavior)

The idea for this approach evolved from the article I specified above. Following snippets helps you to understand the implementation easily. Here we're going to use ClientScript.GetPostBackEventReference with PostBackOptions and Button.UseSubmitBehavior to achieve the resultant.

Snippet 4: (DisableButtonOnClickApp2.aspx.cs)

public partial class DisableButtonOnClickApp2 : System.Web.UI.Page
{
    protected void Page_Load(object sender, EventArgs e)
    {
        DisableButtonOnClick(Button1, "ValGroup1");
    }

    protected void Button1_Click(object sender, EventArgs e)
    {
        StatusMessageLabel.Text = "button 1 click......";
    }

    private void DisableButtonOnClick(Button buttonControl, string validationGroup)
    {
        StringBuilder disableButtonScriptBuilder = new StringBuilder();
        PostBackOptions postBackOptions = new PostBackOptions(buttonControl);

        postBackOptions.ValidationGroup = validationGroup;
        postBackOptions.Argument = "";
        postBackOptions.PerformValidation = true;
        postBackOptions.ActionUrl = "";
        postBackOptions.TrackFocus = false;
        postBackOptions.ClientSubmit = false;
        postBackOptions.RequiresJavaScriptProtocol = true;

        disableButtonScriptBuilder.Append(ClientScript.GetPostBackEventReference(postBackOptions, false));
        disableButtonScriptBuilder.Append("; if(Page_ClientValidate('");
        disableButtonScriptBuilder.Append(validationGroup);
        disableButtonScriptBuilder.Append("')){this.value='Processing...';this.disabled = true;}");

        //buttonControl.OnClientClick = ClientScript.GetPostBackEventReference(postBackOptions, false) + "; if(Page_ClientValidate('" + validationGroup + "')){this.value='Processing...';this.disabled = true;}";
        buttonControl.OnClientClick = disableButtonScriptBuilder.ToString();
        buttonControl.UseSubmitBehavior = false;
    }
}

Snippet 5: (DisableButtonOnClickApp2.aspx)

<%@ Page Language="C#" AutoEventWireup="true" CodeFile="DisableButtonOnClickApp2.aspx.cs" Inherits="DisableButtonOnClickApp2" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
</head>
<body>
    <form id="form1" runat="server">
    <div>
    <table border="0" cellpadding="0" cellspacing="0">
            <tr>
                <td>
                    <asp:Label ID="Label1" runat="server" Text="Label 1 : "></asp:Label>
                </td>
                <td>
                    <asp:TextBox ID="TextBox1" runat="server" ValidationGroup="ValGroup1"></asp:TextBox>
                    <asp:RequiredFieldValidator ID="RequiredFieldValidator1" runat="server" ErrorMessage="RequiredFieldValidator"
                        ControlToValidate="TextBox1" ValidationGroup="ValGroup1"></asp:RequiredFieldValidator>
                </td>
            </tr>
            <tr>
                <td colspan="2" style="text-align: center;">
                    <asp:Button ID="Button1" runat="server" Text="Button" onclick="Button1_Click" />
                </td>
            </tr>
            <tr>
            <td colspan="2" style="text-align: center;">
                <asp:Label ID="StatusMessageLabel" runat="server" Text=""></asp:Label>
            </td>
            </tr>
        </table>
    </div>
    </form>
</body>
</html>

Based on your comfort, use any of the above two approaches.

REFERENCES
http://dailydotnettips.com/2011/02/07/disabling-button-when-performing-some-operation-in-asp-net/#more-482