Tag: azure functions

Scheduling Twitter Content with Azure Functions

A few months ago, I was looking at ways to schedule my Twitter content: basically, I wanted to define when a specific tweet should be posted. The reason behind this? Mostly, I wanted to promote content from my blog for people in different timezones. I live in Singapore, which means that when I use Twitter in the morning, people in Europe are still asleep, and people in the US are likely going to bed soon as well. So I was looking for a way to post the same content multiple times during different times of the day to reach different people. Additionally, I wanted to prepare some content for the monthly CollabTalk, which happens to take place at 1am Singapore time. Not really a time where I plan to be active on Twitter, especially during the week.

While there are nice free and paid tools out there, I wanted to see if I can use Azure Functions for my purpose. With a little help of Azure Table Storage and a 3rd party library, I was able to create the tool fairly quickly, and have been using it for a while now (around 3 months). Here’s how I set it up:

Creating a Twitter application

To post to Twitter as a specific account, we need to set up a new application on https://apps.twitter.com/. After you’ve followed all the needed steps, write down the Consumer Key, Consumer Secret, Access Token, and Access Token Secret.

 

 

Setting Up your Table Storage

We will use Azure Table Storage to store our tweets and their status. I set up a new table called ‘scheduledcontent’ with the following schema:

As for the PartitionKey, I’m using “Scheduled” for any content that still needs to be posted, and “Posted” for content that has been processed already. That way, my queries against the table are quite easy to do, as I can use this column to filter.

Content contains the actual content I want to tweet, scheduledDate the date and time on which it should happen. Lastly, postedDate is for logging purposes, this field will be updated to store the actual date and time of the tweet.

 

Creating your Function App

Moving on, create your Function App. What you need to do here is to go to your App Service Settings and add the twitter keys and secrets from the previous step as key/value pairs:

I named my as follows:

 

Configuring your Function

Now comes to fun part: configuring our Function and adding the relevant plumbing and code. To connect to Twitter and post content, I’m going to use a fork of TweetSharp available at https://github.com/Yortw/tweetmoasharp, and include the corresponding nuget package in my Function.

First of all, we need to include the required Azure Storage and TweetSharp nuget packages. Create a Project.json file and add the following content:

{
  "frameworks": {
    "net46":{
      "dependencies": {
        "WindowsAzure.Storage": "7.0.0",
        "TweetMoaSharp" : "*"
      }
    }
   }
}

Once the packages have been installed, you can configure your Function to trigger on a schedule. I initially had it set to run every 15 minutes, but changed it at some point to run every minute ( 0 */1 * * * *):

 

Let’s move on to configuring the Azure Table Storage connection as an input. Give it a name (inputTable here), enter the Table name you defined earlier, and create the correct Storage account connection.

 

We’ve now finished the configuration, let’s add some code!

using System;
using System.Configuration;
using TweetSharp;
using Microsoft.WindowsAzure.Storage.Table;

public static void Run(TimerInfo myTimer, CloudTable inputTable, TraceWriter log)
{
    log.Info($"C# Timer trigger function executed at: {DateTime.Now}"); 
 
    //fetch all Scheduled tweets
    //Filters: PartitionKey = Scheduled and scheduledDate is in the past
    TableQuery<ScheduledContentEntity> rangeQuery = new TableQuery<ScheduledContentEntity>().Where(
        TableQuery.CombineFilters(
            TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "Scheduled"),
            TableOperators.And,
            TableQuery.GenerateFilterConditionForDate("scheduledDate", QueryComparisons.LessThanOrEqual, DateTime.Now)));
    log.Info($"Fetching all scheduled tweets at: {DateTime.Now}");
    List<ScheduledContentEntity> ScheduledEntities = inputTable.ExecuteQuery(rangeQuery).ToList();
    
    //if there is a scheduled tweet that needs to get posted, let's do it
    if(ScheduledEntities.Count > 0) {
        log.Info($"{ScheduledEntities.Count} new tweets to post at: {DateTime.Now}");

        //authenticating with Twitter
        var _consumerKey = ConfigurationManager.AppSettings["TwitterConsumerKey"];
        var _consumerSecret = ConfigurationManager.AppSettings["TwitterConsumerSecret"];
        var _accessToken = ConfigurationManager.AppSettings["TwitterAccessToken"];
        var _accessTokenSecret = ConfigurationManager.AppSettings["TwitterAccessTokenSecret"];
        var service = new TwitterService(_consumerKey, _consumerSecret);
        service.AuthenticateWith(_accessToken, _accessTokenSecret);

        //post all the scheduled tweets
        foreach (ScheduledContentEntity entity in ScheduledEntities)
        {
            //tweet
            TwitterStatus result = service.SendTweet(new SendTweetOptions
            {
                Status = entity.Content
            });
            log.Info($"{entity.Content} at: {DateTime.Now}");
            log.Info($"{result.CreatedDate.ToString()} at: {DateTime.Now}");
            
            //It was not possible to update table rows, so we have to create a new entry and delete the old one.
            //create new entry
            ScheduledContentEntity content = new ScheduledContentEntity(Guid.NewGuid().ToString(), entity.Content, entity.scheduledDate, DateTime.Now);
            TableOperation insertOperation = TableOperation.Insert(content);
            inputTable.Execute(insertOperation);
            //delete old one
            TableOperation deleteOperation = TableOperation.Delete(entity);
            inputTable.Execute(deleteOperation);
        }
    } else {
        log.Info($"Nothing to tweet at: {DateTime.Now}");
    }   
}


public class ScheduledContentEntity : TableEntity
{
    public ScheduledContentEntity() {         
    }

    public ScheduledContentEntity(string rowkey, string c, DateTime sd, DateTime pd) {
        this.PartitionKey = "Posted";
        this.RowKey = rowkey;        
        this.Content = c;
        this.scheduledDate = sd;
        this.postedDate = pd;
    }
    public string Content { get; set; }
    public DateTime scheduledDate {get; set;}
    public DateTime postedDate {get; set;}
}

This code here should be pretty straight-forward to read and understand. Every minute, it checks if there are any entries in our table storage which are set as ‘Scheduled’ (PartitionKey), and if so, tweets them.

 

Creating an extremely fancy ultra-modern frontend

Well, not really. Sorry, nothing great to show here, the backend was more important to me than the frontend. I created an extremely simple MVC app with a form that creates new entries in my Azure table (RowKey is a random GUID, by the way), and shows scheduled tweets in a standard table on the page. I also use Azure Storage Explorer from time to time, mostly for editing. I’ll leave you up to you to create a fancy site with all the features you want – URL shortening, editing of scheduled tweets, …

 

The Result

As I mentioned, I’ve been using this set up for some time now, but only scheduled around 50 tweets so far. In case you’re wondering how fast it is, most of the time the tweet gets posted 1-2 seconds after I scheduled it:

Any errors so far? None:

 

I was also asked to provide some information on the costs. Here are the costs for the past 30 days:

 

What’s Next?

For my purpose, this is already good enough. While it lacks a lot of things (as mentioned, editing of tweets, shortening of URLs, error handling, maybe a bit more security), it does its job quite well, so I’ll keep it as it is for the moment

Using Azure Functions to receive daily Website Updates via Email

Note: In this blog post, I’m talking about Azure Functions, which are currently in preview. Settings, functionality, pricing, etc. may still change.

Introduction

I’m constantly trying to optimise my life by automating small tasks, so that I can either save some time and get things done without having to remember them. For example, I’m a big fan of both IFTTT and Instapaper, and use these two in combination to have new blog articles (e.g. from dev.office.com/blogs) posted into my Instapaper account automatically, so I can’t read those easily and quickly when I’m travelling to and from work. I do not need to check on a daily basis for new articles, and I can start reading new ones the moment I’m on public transport.

Recently, I started looking at Azure Functions. If you’ve never heard of them before, I recommend starting with the official Azure Functions Overview for an introduction, followed by Scott Hanselman’s and Troy Hunt’s recent blog post on potential use cases. After playing around for a short while with them, I thought I’d give them a chance for a mini-project: I’ve been following Packt Publishing’s Free Learning eBook offer for a while now. When I say follow, I actually mean I open my browser, open the page, and check which new eBook is available.

Pack Publishing Free Learning eBook Offer

This obviously takes some time, and additionally I forget to do it on a daily basis. While I do not get every free daily eBook, I still want to know if there’s something interested published. So, what I was looking at doing was to implement a small website monitor that checks the offer page once per day and sends me an email with the offer details. Quite simple, should be up and running within a short time.

 

Preparation

First of all, you obviously need an Azure account. In addition to that, you need a SendGrid account to send out emails. Luckily, Azure subscribers can get a free SendGrid account with up to 25,000 emails each month. More than enough for personal use! Within Azure, click on New, type in SendGrid, and follow the steps to add it:

Azure SendGrid

Once done, create a new API Key in SendGrid and keep it somewhere (as we will need it later).

Furthermore, you also need to add a Function app in Azure if you haven’t already done so. When you create a Function app, you need to specify whether to use a Dynamic App Service Plan or a Classic one. For our small exercise, Dynamic is the best choice (“You don’t have to worry about resource management, and you only pay for the time that your code runs.“). Read more about the differences on How to scale Azure Functions.

Note: You can review the pricing for Functions here (note: Functions are still in Preview, pricing may change). You can also estimate your total cost by using the Azure Pricing calculator. Here’s my estimate for the Function I’m talking about:

azure-functions-pricing

Yes, you can safely run your Function for quite a couple of times before getting charged for it.

 

Once both the Function App and the SendGrid API Key are available, you need to add the key to the Function App’s settings. Open your Function App, go to Function app settings, and select Configure app settings.

azure-functions-app-settings

Under App settings, add a new setting with AzureWebJobsSendGridApiKey as key and your SendGrid API Key as value.

azure-functions-app-settings-sendgrid-api

Once that’s done, save your changes. Your Function apps are now capable of sending out emails via SendGrid.

Creating the Function

In your Function App, create a new Function. Microsoft provides several templates, we’ll use the simplest one and do all the plumbing from scratch (but feel free to check out the available templates to learn how you can integrate for examples Azure Queues or Azure Blob Storage), so select the “Empty – C#” template and give it a name.

Once created, each Function offers you 4 pages where you can manage it – Develop, Integrate, Manage, and Monitor.

azure-functions-pages

Let’s start with setting up the basic “structure” of our Function on the Integrate page. Initially, as we’ve chosen the Empty template, there are no Triggers, Inputs, or Outputs defined. What we want to set up is a Timer trigger (to run our code on a daily basis) and a SendGrid Output (to inform us about the latest offer):

azure-functions-integrate

First, add a new Trigger and select the Timer trigger template. Set the Schedule to 0 0 0 * * *. which means that the Function runs every day at midnight (once you add a Timer trigger, the page will also provide some documentation on the format for this in case you want to define your own schedules). Remember to click Save.

azure-functions-integrate-timer

Next, add a SendGrid Output. You have the possibility to configure a couple of required parameters either on the configuration page or in your code. Leave the Message parameter name as it is (this is the variable we’ll use in our code), and define the other values as required. As I’m sending the email to myself, I added my email address to both the To and From fields. The Subject and Body will get defined in the code.

azure-functions-integrate-sendgrid

Save again, and click on Develop. At the top of the page you’ll see the Code section where you can define the underlying code of your Function.

azure-functions-develop-code

Copy the following code sample and replace any existing code in your Function:

#r "SendGridMail"

using System;
using SendGrid;

 static string packtUrl = "https://www.packtpub.com/packt/offers/free-learning/";

 public static void Run(TimerInfo myTimer,  out SendGridMessage message, TraceWriter log)
 {
     message = null;    
     try
     {
        using (var httpClient = new HttpClient())
         {
             string pageContent = httpClient.GetStringAsync(packtUrl).Result;
             //the content we're looking for comes after a div tag with the class dotd-title
             int tmpIndex = pageContent.IndexOf("dotd-title");
             string offerDetails = pageContent.Substring(tmpIndex +15, 3000);
             //we've now got more content than required, removing anything that is not part of the article description
             offerDetails = offerDetails.Substring(1, offerDetails.IndexOf("dotd-main-book-form")-20);
             offerDetails += String.Format("<br/><br/><a href='{0}'>PacktPub Free Learning</a>",packtUrl);
             //parsing the book title here
             string bookTitle = offerDetails.Substring(offerDetails.IndexOf("<h2>")+4,
                                 offerDetails.IndexOf("</h2>")-9).Trim();
             
             message = new SendGridMessage()
             {
                 Subject = String.Format("Packt's Free Learning Offer: {0}",bookTitle),
                 Html = offerDetails
             };

             log.Info("Finished successfully");
         }
     }
     catch (Exception exc)
     {
         log.Info("Exception:" + exc.Message + exc.StackTrace);
     }
 }

Note the following:

  1. The first line (starting with #r) is used to add a reference to an external assembly (documentation)
  2. public static void Run is the main function which gets called when your Function runs. As you can see, we have 3 arguments for it here. TimerInfo myTimer needs to be added as we’re using a Timer trigger. out SendGridMessage message is the SendGrid email output which we defined in the Integration configuration. We need to instantiate it and set any required properties if we want to send an email. Lastly, TraceWriter log provides a mechanism to log any information that you want during your code execution – errors, warnings, general stuff. Anything written to this log gets displayed in the Log section just below your Code section.
    azure-functions-develop-logs
  3. As for the code fetching and extracting the information I want to receive via email, this is pretty straightforward. The few lines I have here instantiate a new HttpClient object, read the offer page content, and fetch the title and description (not in the nicest of ways, but hey, it’s working!).
  4. Lastly, we’re creating our SendGrid email message and assign it the appropriate subject and email body.

Here’s the final result once the code was executed (either when and the email was sent:

azure-functions-packtpub-email

 

That’s it! A couple of minutes of configuration and coding, and I now receive daily email alerts about the latest free eBook.