Using ML.NET for Predicting Hourly User Session Counts

Harshal Sonparote 1 Reputation point

I am new to AI/ML and need assistance with a business scenario that involves machine learning, specifically using ML.NET as our preferred technology.

We have collected hourly user session count data per country for the last year through log analytics. The goal is to train a model using this data to predict hourly user session counts for future dates.

Could anyone suggest a suitable model for this use case? We've explored time series and regression models, but we're currently struggling to achieve 24-hour predictions for specific future days.

Any sample code or resources would be greatly appreciated.

0 comments No comments
Sign in to comment

1 answer

  1. Sina Salam 30,166 Reputation points • Volunteer Moderator

    Hello Harshal Sonparote,

    Welcome to the Microsoft Q&A and thank you for posting your questions here.

    I understand that you would like to know how you can use ML.NET for Predicting Hourly User Session Counts.

    You are on the right track and welcome to the world of AI/ML! Your use case / scenario of predicting hourly user session counts is a classic time series forecasting problem, and the below are some suggestions and resources to help you achieve your goal with ML.NET:

    1. For predicting hourly user session counts, a time series forecasting model is appropriate. In ML.NET, you can use the Singular Spectrum Analysis (SSA) algorithm, which is well-suited for univariate time series data. SSA can help you capture trends, seasonality, and noise in your data.
    2. To achieve 24-hour predictions for specific future days, you can use the SsaForecastingEstimator in ML.NET. This estimator allows you to forecast multiple steps ahead, which is ideal for your requirement of predicting hourly counts for an entire day.
    3. The sample code as requested:
      1. Open Visual Studio and create a new C# Console Application.
      2. Install the necessary NuGet packages: Microsoft.ML, Microsoft.ML.TimeSeries.
      3. Ensure your data is in a suitable format, with columns for the timestamp and the session counts.
      4. Load your data into the application.
      5. Define Input and Output Classes: 
         public class ModelInput
 {
 public DateTime Timestamp { get; set; }
 public float SessionCount { get; set; }
 }
 public class ModelOutput
 {
 public float[] ForecastedCounts { get; set; }
 public float[] LowerBoundCounts { get; set; }
 public float[] UpperBoundCounts { get; set; }
 }
      6. Create and Train the Model: 
         var mlContext = new MLContext();
 var dataView = mlContext.Data.LoadFromTextFile<ModelInput>("data.csv", hasHeader: true, separatorChar: ',');
 var forecastingPipeline = mlContext.Forecasting.ForecastBySsa(
 outputColumnName: nameof(ModelOutput.ForecastedCounts),
 inputColumnName: nameof(ModelInput.SessionCount),
 windowSize: 24,
 seriesLength: 365 * 24,
 trainSize: 365 * 24,
 horizon: 24);
 var model = forecastingPipeline.Fit(dataView);
      7. Make Predictions: 
         var forecastEngine = model.CreateTimeSeriesEngine<ModelInput, ModelOutput>(mlContext);
 var forecast = forecastEngine.Predict();

    For more reading and resources use the following links:

    I hope this is helpful! Do not hesitate to let me know if you have any other questions.

    Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.

    1. Harshal Sonparote 1 Reputation point

      Thank you @Sina Salam for quick response.

      I have few queries on suggested approach . Can you please assist.

      1. Does This approach covers predicting numbers based on yesterday's user session count at given time e.g. (11 AM), same time / same day last week , same time/ day last two weeks back , same time/date last year , holidays etc. We want to make sure we are on right track for these future scenarios.
      2. As I can see , using SsaForecastingEstimator we can get predictions for next X hours predictions. Is there a way to get predictions for specific day (eg. prediction for a day which is after 72 hours)
      3. Also , In ML.NET , can we use ARIMA for getting predictions . Can you please recommend if ARIMA is best suited over SSA for my business scenario.

      thank you for your support on this matter.

    2. Harshal Sonparote 1 Reputation point
      • @Sina Salam - The most important use case here is the user session count should be predicted based on day, weekend ,time from historical data. so i do see there is relationship of count to time and day of the week
    3. Sina Salam 30,166 Reputation points • Volunteer Moderator

      Hello Harshal Sonparote,

      Thank you for your feedback.

      Regarding your comment and questions:

      Does This approach covers predicting numbers based on yesterday's user session count at given time e.g. (11 AM), same time / same day last week , same time/ day last two weeks back , same time/date last year , holidays etc. We want to make sure we are on right track for these future scenarios.

      In ML.NET, Singular Spectrum Analysis (SSA) captures trends and seasonality in time-series data, but it is limited in handling multiple types of seasonal patterns and external variables like holidays directly. Here’s a breakdown:

      • SSA can handle basic patterns and seasonalities, such as daily and weekly trends. However, it might struggle with complex seasonalities like annual trends or holiday effects since it does not inherently support exogenous variables (additional contextual information).
      • For these scenarios, consider adding features manually that reflect these patterns:
      • Include columns for day of the week, hour of the day, holiday flag, etc., as part of your input data.
        • If the patterns you’re targeting (e.g., same time last year) have significant impact, adding more historical data to your time series can help the model learn these periodic trends.

      As I can see , using SsaForecastingEstimator we can get predictions for next X hours predictions. Is there a way to get predictions for specific day (eg. prediction for a day which is after 72 hours)

      With the SsaForecastingEstimator, you can specify the number of future steps to predict (horizon). However, there’s no built-in method to predict for arbitrary future points, such as a specific day and hour far ahead. Here's how you might handle this:

      • If you want predictions further out, set the horizon parameter to cover that timeframe (e.g., 72 hours for a 3-day forecast).
      • You could also use recursive forecasting, where you iteratively use previous predictions as inputs for extended forecasting, though this can compound errors over longer time horizons.

      Also , In ML.NET , can we use ARIMA for getting predictions . Can you please recommend if ARIMA is best suited over SSA for my business scenario.

      ARIMA models are widely used for time series forecasting and handle seasonal trends better than SSA, especially for data with clear seasonality (daily, weekly, etc.). Here are considerations for both:

      • Unfortunately, ML.NET does not currently support ARIMA. However, you could consider ML.NET together with a Python integration (e.g., Microsoft.ML.Python), or use a custom-built ARIMA model in Python and export predictions for integration with ML.NET.
      • SSA is generally suited for univariate data and handles simple patterns, while ARIMA is better for capturing time-based dependencies with seasonal patterns. Given your need for day, time, and weekend relationships, ARIMA (or a seasonal extension like SARIMA) may be a better fit.

      The most important use case here is the user session count should be predicted based on day, weekend ,time from historical data. so i do see there is relationship of count to time and day of the week.

      To account for variations by day and weekend patterns, adding custom features to your data could help improve model accuracy. Here’s how you can do this in ML.NET:

      • Include fields such as DayOfWeek, IsWeekend, HourOfDay, and any other custom patterns your data displays.
      • ML.NET's SSA doesn’t directly use these features, but manually augmenting your dataset can help it capture time-based trends.

      Success.

    4. Harshal Sonparote 1 Reputation point

      Apologies for too many questions. Now that you are fully aware of my use case , I would like to get your opinion on final comments

      1. SSA does not sound like a good fit as we are dealing with complex seasonality.
      2. ARIMA can be a good fit if we can use this in ML.NET . Can you please share if there is any sample for ML.NET together with a Python integration (e.g., Microsoft.ML.Python).
      3. I am not sure if that's correct approach , However I have created a POC for regression algorithm using ML.NET and used fastTree algorithm using ML.NET and used features like time , day , holiday etc . The predictions are nearly correct. Can you please share your recommendation for this approach.
    5. Sina Salam 30,166 Reputation points • Volunteer Moderator

      Hello Harshal Sonparote,

      Thank you for your feedback.

      Given your use case involving complex seasonality and the need to predict user session counts based on various time-related factors, Singular Spectrum Analysis (SSA) in ML.NET may not be the best fit. ARIMA models, which handle seasonal trends better, could be more suitable, but ML.NET does not natively support ARIMA. You can integrate Python with ML.NET using NimbusML to leverage ARIMA models - you can find sample here: https://github.com/Microsoft/NimbusML-Samples

      Your approach of using a regression algorithm with features like time, day, and holidays in ML.NET is promising, especially since your proof of concept with the FastTree algorithm has shown nearly correct predictions. This method can effectively capture the relationships in your data and provide accurate forecasts. https://github.com/Microsoft/NimbusML

      Success

    Sign in to comment
Sign in to answer

Your answer