Be Creative

Creativity isn’t necessarily painting happy trees or composing the next symphony but we live our lives in a world that offers a wide range of ways to be creative.  Doing things, the same old way, may be comfortable but the same old way can be boring.  Humans can find their greatest joy in coming up with their own way to accomplish a task.

We can be creative with everything we do.  Something that I did a long time ago was to add a flower petal on a console application that was doing data processing.  I needed to know that the application was continuing to process data and I could have done this any way, but I thought it would be good to do this with a flower petal.  There was more time involved with showing progress this way but it was well worth it to me for my thought processes. 

Creativity for me is finding cool ways to do something sometimes just changing it up a bit by using different methodology to solve a problem or writing a blog post to help others.  This is the first post in a series of posts that outline how I have become successful as a developer.  I will post these over the course of the next couple of weeks and at the end will give you a cool acronym to go along with this series (having some creative fun). 

It's PI Day...

PISo it is PI Day and I thought I would write something up about PI.

First, what is PI?  PI is a number that is the exact ratio between the diameter and the circumference of a circle or the distance across to the distance around.  PI is an irrational number meaning that its precise number in decimal notation is infinitely long and will never repeat.  So PI is the best way to get a glimpse into the mind of God and how He designed this grand part of the world.

Second, PI... is it 3.14 or is it 3.14159 and what does that matter?  This comes into accuracy and precision.  Accuracy is the closeness of a measured value to a standard, think of hitting a bullseye.  Precision is the closeness of measurements to each other, think of grouping all of the hits on a bullseye near each other but not on the bullseye.  When a measurement is both precise and accurate then it will would be the center of a bullseye.  The greater number of decimals used with PI would increase the accuracy of the number.

So it would appear that using a more accurate number of PI would be beneficial, but when you start using highly accurate numbers with computers issues with data types begin to come into play.  Depending on the data type, the processor on the computer will have optimization for doing the calculations.  Float for example is used for high performance but isn't accurate.  This would mean that even if a highly accurate number with PI is used with a float will not generate precise or accurate numbers.  

So how accurate is too accurate?  Well, suffice to say there is a certain point of diminishing returns for using a highly accurate number of PI.  So according the the Jet Propulsion Lab, fifteen decimal places will calculate the a circle that is 78 billion miles around with the accuracy of 1.5 inches.  If PI with fifteen decimal places is used to calculate a circle that can fit on the earth, it will have an accuracy to within one molecule.  Forty-six decimal places of PI is able to calculate a circle to an accuracy of less than the diameter of a hydrogen atom for a circle that would just fit within the entire visible universe.

So the reality of the world is, 3.14 is a great number and a precise number for every day life.

inotify Instances .Net Core on Linux

Now you get the exception "The configured user limit (128) on the number of inotify instances has been reached" and you wonder what the heck is going on.  For me I never got this exception on my local instance of Docker running on my Mac or in Windows but I would see my app just stop running in Linux.  Even weirder I could run the app fine in Windows hosted by Kestrel.  So where does this error start coming from?  

After much digging, I found the problem was with the IConfiguration interface.  Specificially, I was calling pulling the JSON configuration files multiple times which was reloading the IConfiguration interface.  The issue was that it was rebuilding the configuration and re-attaching the "reloadOnChange" setting.  Since the reloadOnChange was set to true it was attaching a new file watch multiple times and was hitting the limit.  That all being said, I never saw this issue locally on my Mac or Windows or my local instances of Docker on either which I think is related to the way the underlying Docker Linux host manages the watch on the files.  

Something else as a side note, since the files were being tracked, even though I was disposing of the class it wasn't releasing the watch and thus keeping these connections open.  The simple fix was to set the reloadOnChange to false but to make it even better was to just create a singleton of the data and thus prevent the data from being ever reloaded.  I don't think it is necessarily a bug with .Net but it is a weird side effect of how the host can cause problems with a framework.  

I hope this helps some people save some time that I lost dealing with this crazy issue.

 

Sending Mass Emails in C#

I like many developers am tasked with creating batch emails to send to a large population of users.  And for this I am not talking about spam, I am referring to sending emails to a large poplulation within an organization such as annual notification reminders or custom links to some application.  I have tried sending these emails synchroniously and if you are sending a few hundred that isn't a problem, but when I started sending several thousand, I was noticing a bottleneck with my SMTP server and waiting for the callback showing the SMTP server receved the message.  

So moving forward from that thought the best option for sending mass emails greater than a few hundred is using threading and asynchronious sending.  I have tried this a few different ways in the past with event listeners and different techniques, but moving to the async and await world of modern .Net I thought I would re-write it with some more modern tooling.  Since this took me a little trial and error I thought it would be a good thing to post for my fellow developers out there. 

From a performance standpoint using the same SMTP server I was getting about three emails per second with standard synchronous methods and when I went to the attached code, I went up to about 18 (running locally over VPN to our corporate offices) and I will expect to see an even bigger jump on a VM in the datacenter.  Overall there isn't much impact to performance on the machine with minimal CPU and RAM overhead.  

Anyway, click here to get to my code.  

Two things I want to point out:

  1. I know I have a weird try / catch in there.  That is because the mail server was choking with that many emails going through at one time and I was getting random errors.  Retrying one time usually fixed it.
  2. I know there are async methods with the SMTP client, but when you use that it still only allows one email to be processed at a time by the general SMTP client thus making it an async method but not async in the since that I wanted which was mass emails with threading.