To preview this task, watch the video beginning at 00:58.

This tutorial uses a sample project that contains the data sets, notebook, and prompt templates to perform the analysis. Follow these steps to create a project based on a sample:

1. Access the Stock anomalies analysis project in the Resource hub.

   1. Click Create project.

   2. Accept the default values for the project name, and click Create.

   3. Click View new project when the project is successfully created.

2. Associate a Watson Machine Learning service with the project:

   1. When the project opens, click the Manage tab, and select the Services and integrations page.

   2. On the IBM services tab, click Associate.

   3. Select your Watson Machine Learning instance. If you don't have a Watson Machine Learning service instance provisioned yet, follow these steps:

      1. Click New service.

      2. Select Watson Machine Learning.

      3. Click Create.

      4. Select the new service instance from the list.

   4. Click Associate service.

   5. If necessary, click Cancel to return to the Services & Integrations page.

3. Click the Assets tab in the project to see the sample assets.

For more information or to watch a video, see Creating a project.

For more information on associated services, see Adding associated services.

To preview this task, watch the video beginning at 01:27.

The three data sets in the sample project contain synthetic data generated using public stock data from the Yahoo! Finance website as a basis. The training data for a time series anomaly prediction model must be structured and sequential. In this case, the synthetic data is structured and sequential. Follow these steps to view the data assets in the sample project:

1. Open the historical_data.csv data set. This data set contains historical stock price performance from May 2012 to May 2016.
2. Return to the project's Assets tab, and open the test_data.csv data set. This data set contains historical stock price performance in Q1 2023.
3. Return to the project's Assets tab, and open the training_data.csv data set. This data set contains historical stock price performance in 2023.
4. Click the Visualization tab.
   1. Select the Date column, and then click Visualize data. The first suggested chart type, a histogram, displays.
   2. Select the Line chart type.
      1. For the X-axis, select the Date column.
      2. For the Y-axis, select the Adj Close column. This shows the adjusted closing price by date. The target column for anomaly analysis is the adjusted closing price.

To preview this task, watch the video beginning at 02:13.

You can use a variety of tools, such as, AutoAI, SPSS Modeler, or Jupyter notebooks to train the model. In this tutorial, you will train the time series analysis anomaly prediction model with AutoAI. Follow these steps to create the AutoAI experiment:

1. Return to the project's Assets tab, and then click New asset > Build machine learning models automatically.

2. On the Build machine learning models automatically page, type the name:

   Stock anomaly experiment
   

3. Click Create.

4. On the Add data source page, add the training data:

   1. Click Select data from project.

   2. Select Data asset > training_data.csv, and click Select asset.

5. Set the time series analysis settings:

   1. Select Yes if you are asked to create a time series experiment.

   2. Select Anomaly prediction.

6. Select Adj Close for the Feature columns.

7. Click Run experiment. As the model trains, you see an infographic that shows the process of building the pipelines.
   

   For a list of algorithms, or estimators, available with each machine learning technique in AutoAI, see: AutoAI implementation detail.

8. After the experiment run is complete, you can view and compare the ranked pipelines in a leaderboard.

   

9. You can click Pipeline comparison to see how they differ.

   

10. Click the highest ranked pipeline to see the pipeline details.

11. Review the Model evaluation page to see the detailed evaluation metrics about the model performance.

    The AutoAI tool considers a wide range of criteria to spot anomalies. In the table, you can see the evaluation based on different metrics, such as Average precision and Area under ROC, for each of the anomaly types.

    Anomaly types

    Anomaly Type	Description
    Trend anomaly	A segment of time series, which has a trend change compared to the time series before the segment.
    Variance anomaly	A segment of time series in which the variance of a time series is changed.
    Localized extreme anomaly	An unusual data point in a time series, which deviates significantly from the data points around it.
    Level shift anomaly	A segment in which the mean value of a time series is changed.

12. Save the model.

    1. Click Save as.

    2. Select Model.

    3. For the model name, type:

       Anomaly Prediction Model
       

    4. Click Create. This saves the pipeline as a model in your project.

13. When the model is saved, click the View in project link in the notification to view the model in your project. Alternatively, you can navigate to the Assets tab in the project, and click the model name in the Models section.

To preview this task, watch the video beginning at 03:40.

The next task is to promote the test data and the model to a deployment space, and then create an online deployment.

To preview this task, watch the video beginning at 05:07.

Although the Prompt Lab can work with structured and unstructured text, it is essential to ensure that you input the right data that the model can process. In this use case, you need to process news article text based on the anomaly dates you obtained from the anomaly prediction. You can integrate an external news API to extract the relevant news during those dates to simplify the data-gathering process. You can do this in a Jupyter notebook with Python code.

Since the foundation models have a limit on the number of tokens they can process in a single prompt (known as the context window), data may need to be chunked or summarized to fit within this limit. This step ensures that the input data is in a format that the foundation model can effectively process without losing essential information.

Follow these steps to run the notebook:

1. From the Navigation Menu , choose Projects > View all projects.
2. Open the Stock anomalies analysis project.
3. Click the Assets tab.
4. Click the Overflow menu for the Extract and Chunk Text from News Articles notebook, and choose Edit.
5. Complete the Setup section.
   1. Click the Run icon for the first cell to import the libraries.
   2. Obtain the necessary API keys:
      1. Follow the link to create an account and API key at TheNewsAPI.
      2. Paste the API key in the thenewsapi_key variable.
      3. Follow the link to create an account and API key at ArticlExtractor.
      4. Paste the API key in the extract_key variable.
   3. Run the cell to set the two API key variables.
6. Run the cells in the Define the function to get news article URLs section.
   • The first cell defines a function to get data from TheNewsAPI's Top Stories and set up parameters to ensure you can get relevant news.
   • The second cell defines a function to only get a list of URLs based on the response.
7. Run the cells in the Define the function to extract news text section.
   • The first cell defines a function to extract news text from a specific news URL using ArticlExtractor API.
   • The second cell defines a function to combine news text from all of the article URLs obtained from TheNewsAPI.
8. Run the cell in the Define the function to chunk news text section. To ensure the LLM foundation model can take on the information from the text, you need to make sure the token doesn't exceed the context token window limitations. In this example, you define a function to use LangChain to split the character text by taking into account the context of the news text.
9. Run the cell in the Execute the functions section. In the response, you can see that the final output of data is ready to be fed into the Prompt Lab. LangChain’s text splitter splits the long text up into semantically meaningful chunks, or sentences, and combines them again as a whole text to be processed. You can adjust the maximum size of the chunks.

To preview this task, watch the video beginning at 07:17.

Now that you have the relevant news article appropriately chunked, you can construct your own prompt templates in the Prompt Lab, or you can use the sample prompt templates in the sample project. The sample project includes sample prompt templates for summarization and question answering tasks. Follow these steps to prompt the foundation model in the Prompt Lab.