Formulating and running a model: house construction scheduling
This tutorial shows you how to use the Modeling Assistant to define, formulate and run a model for a house construction scheduling problem. The completed model with data is also provided in the DO-samples, see Importing Model Builder samples.
You need to plan and schedule activities and subcontractors for a house construction project. Your schedule must start on a particular date. All the activities (masonry, carpentry, plumbing and so on) must be scheduled and there is a specified order of activities that must be respected (for example windows cannot be put in until the roof is completed). Each subcontractor can perform some of the necessary activities and with differing level of skills. Your schedule must determine the best (earliest) end time for the construction project ensuring that all activities have been scheduled and decide which subcontractor to assign to each activity. In addition, you would like to know how to optimize the skill level of your subcontractors on this project.
You have data for this project as shown in the following spreadsheet. For each activity you have the duration that is needed to complete it, the activities that must precede it and the possible subcontractors who are available and qualified to perform that activity.
For illustration purposes, there are just 10 activities and 3 subcontractors shown. With Decision Optimization it is easy to change your data and solve the same problem with larger data sets.
For each activity you also have data concerning the level of expertise that each subcontractor has for that activity. The higher the number, the more expertise the subcontractor has. If a subcontractor has a zero skill level, he must not be assigned to the task. The following table shows part of this spreadsheet.
You also have a table containing the names of the Subcontractors (Joe, Jack and so on) available for this project.
Obtain data files for this example
The data files used in this example are available in the DO-samples. Normally, you would have your files already stored in Watson Studio in your own data asset or locally on your machine, For illustration purposes however, so that you can build the model yourself, in this example you will first download the data files onto your machine and then import them into the project that you have just created. The completed model formulation with imported data is also provided as a sample, see HouseConstructionScheduling in DO-samples.
- Download and extract all the DO-samples on to your machine.
- Open your project in IBM Watson Studio.
- Click Add to Project.
- Select Data.
- Browse to locate the
house_subcontractor.csvin the datasets folder selecting the relevant product and version subfolder in your downloaded DO-samples.
- Click Open. The files are uploaded as data assets in your project.
Create a Scenario
- Add a Machine Learning service to your project. You can either do this at the project level (see Creating a Watson Machine Learning Service instance), or this can be done when you first create a new Decision Optimization experiment: click Add a service, select or create a New service, click Associate service in the item bar, then close the pane.
- Associate a deployment space with your Decision Optimization experiment (see Deployment spaces). A deployment space can be created or selected when you first create a new Decision Optimization experiment: click Create a deployment space, enter a name for your deployment space and click Create. For existing models, you can also create or select a space in the Overview information pane.
To create a Scenario:
- Create a project in IBM Watson Studio. Select Create an empty project, enter a project name and click Create.
- In the Overview tab of your project, click add a Machine Learning service and select an existing service instance (or create a new one) and click Select.
- Click Add to Project.
- Select Decision Optimization experiment.
- In the New Decision Optimization experiment window that opens, enter a name.
- Choose a deployment space from the drop-down menu (or create one) and click Create. If you haven't already associated a Machine Learning service with your project, you must first select Add a service to select or create one, before choosing your deployment space for your experiment.
- Click Create. A Scenario 1 is created along with the model, and you work in Scenario 1.
Your Scenario specifies the combination of data and the optimization model formulation that you want to solve. You can create different scenarios with different variants of data and model formulations.
The model builder opens displaying the Prepare data view. The data files that you have in your project are displayed in the data pane. (If necessary, click to open the data pane.) Select the three house sample files and click Import.
The data files you imported are now displayed as tables in the Prepare
view. The following image shows the data files
house_subcontractor.csv imported in a Scenario.
You can view all the data by scrolling in a table. You can also view all the data by clicking the Open the table in full mode icon of a particular data table. You can edit data values directly in the table as well as in full mode.
Choose the Modeling Assistant
- Click Run model in the sidebar and a pop-up window appears asking you how you want to formulate your model (whether you want to create or import a model with Python or OPL, or use the Modeling Assistant).
- Select Use modeling assistant.
Define your decision domain
In the Model view: select the decision domain for your problem. In this case select Scheduling. The decision domains currently defined are Scheduling, Resource Assignment, Selection and Allocation and Supply and Demand domains.
- After selecting your domain, a pop-up window appears for you to map your data to the scheduling concepts Tasks and Resources. Tasks are whatever you want to plan and schedule over time. You must define at least one task to be scheduled. In this example, your tasks are construction activities such as masonry. Resources can be human, machine, equipment or anything you want to use for the tasks. In this case your resources are your subcontractors.
- Under TASKS, click Choose a task and choose
house_activityfrom the drop-down list. Then under RESOURCES click Choose a resource and choose
house_subcontractor. The names of possible tasks and resources for you to choose from are taken from your imported data. For this example, you only need to map activities and subcontractors, but you could add other tasks and resource mappings if your model required it. You can remove any mapping by hovering over it and selecting the delete icon to its right.
- Click Continue.
How tasks will use resources
In the window, for each task to be scheduled, you have three options :
- Use resources with assignment: You can select all the options and choose to have your activities assigned to specific subcontractors. This means that you want to obtain a schedule for your house construction activities with the best sequence of house construction activities, taking into account the start times, durations and precedence order, and so on, and with named subcontractors assigned to the activities. This option is selected by default.
- Use resources without assignment: You can use resources, and clear the While
assigning... check box to choose not to assign specific contractors to your activities. This
means that you want to obtain a schedule for your house construction activities with the best
sequence of house construction activities, taking into account the start times, durations and
precedence order, and so on. You still want the numbers and types of subcontractors you have
available to be considered in the obtained schedule (for example 3 plumbers, 2 carpenters,... ), but
they don't have to be assigned to specific people (for example Joe, Jack, Jim).
When you use resources, with or without assignment, you can also decide to add further time-based capacity constraints to your model. For example you can specify limits on the number of subcontractors that can be used in parallel at any given time, or individual or total subcontractor availability over a time period.
For an example of scheduling without assignment, see BridgeScheduling in DO-samples.
- Continue without resources You can clear all the options and just click Continue to schedule the tasks ignoring all resource limits. This means that you want to obtain a schedule with the best sequence of house construction activities, taking into account the start times, durations, precedence orders, and so on, but without considering your subcontractors.
For this example:
- Choose the default setting with all options (Use the resources...
While assigning...) selected and click Continue.
The problem that you want to solve is now formulated in a concise statement.
- Click Finish.
You return to the Model view, and a guided tour opens that you can either follow or close. You can edit your problem definition again at any time, by clicking the Edit intent (pencil) icon and redefining your mappings and scheduling options.
Your model formulation
Now that you have specified the problem that you want to solve, the Modeling Assistant provides you with a partially completed formulation in this model view. The Objectives and Constraints pane contains the model that you will run. The Add to model pane displayed on the right, contains more suggestions that you can include in your model formulation. If you have re-sized your window, it is possible that the Add to model pane appears underneath the Objectives and Constraints pane.
The model consists of an objective to be attained (maximized or minimized) and some constraints that must be satisfied. For scheduling problems like this, your objective is to work out the best schedule. The best, in this case, is one in which the time to complete all the activities is minimized. (You want to complete the house construction as quickly as possible as this will reduce costs.) This objective as well as some standard scheduling constraints have been automatically added to your model. You can also use the search field above the Objectives to search the objectives and constraints.
- the scheduling will be performed from the start time that you define for your construction project
- each subcontractor can only be assigned to one task at a time.
- each activity has one subcontractor assigned to it
- all activities are present in the schedule, in other words, no activity can be omitted from the schedule
- the duration time for each activity is respected
It is possible that your constraints are displayed in a different order. There is also a constraint that is automatically added to all scheduling problems with assignment. This enables you to accept or refuse to assign subcontractors who have unavailable periods during the activities that are scheduled. In this example, unavailable periods are not considered so leave this constraint as it appears by default.
Some constraints have more details that can be displayed or hidden by clicking the arrows on each line. A bar next to the constraint indicates that there is a value or definition that you must add. You can add items by clicking the term shown underlined and typing in or selecting from a drop-down list, and you must complete the model before running it, but before doing this, first save a copy by duplicating the scenario as explained later in this section.
In the model view of your scenario, if you click the Replace arrow next to Modeling Assistant, you will return to the screen where you choose whether you want to create your model in Python or OPL, with the Modeling Assistant or in a Python notebook or import an existing model. If you choose to replace your model at this stage, you will overwrite your current model and lose your changes. If you want to keep a copy of your current work in progress, create a new scenario before changing the model.
Duplicate the scenario
To keep a copy of this model, make a copy of this scenario:
- If the scenario pane is not open, click the Scenarios icon.
- Click the three dots next to Scenario 1 and select Duplicate.
- Enter a name for the new scenario, Scenario 2, for example, and click Create. You continue working in Scenario 2.
Complete your model
- If necessary, expand the duration constraint by clicking the arrow on this line to display the
full definition. Select must be
and choose the column name
Duration in days. The default duration unit expressed in default duration unit is added to the end of the constraint. You might modify this by clicking default duration unit and selecting days, but the default unit is days. You can also modify the default duration unit and customize how dates and times are defined, in the Settings panel. Once you have completed the duration constraint, the row is no longer highlighted.
- In the Schedule start constraint, click the date displayed. Then enter a date (or a date and time) and select this from the drop-down menu to replace the currently displayed date. If you enter a date without a time, the default time is taken to be 00:00.
The constraints are no longer highlighted once you have entered values. The model, however, isn't quite complete. You might want to make sure that your schedule takes into account the order of precedence of tasks so that each activity can only start after those that must precede it. You will add this constraint later.
If your model had more objectives and constraints, you could browse or filter them by using the Find in my objectives and constraints search field.
You can choose to Disable or Remove any one of the objectives or constraints in your model by clicking the 3 vertical dots next to the statement. This menu also enables you to reorganize the order of your statements by moving them up and down and you can also duplicate a statement.
When you have completed your model, or when there are no objectives or constraints still highlighted, you can run it to find a solution that will decide the best optimal schedule based on your model objectives and constraints.
Run your model
You can change the solve time limit for your model in the Run model view by clicking the Settings tab next to the suggestions. For this example, use the default limit. Other parameters can also be set using run configuration parameters (see Run parameters for more information).
In Scenario 2, click the Run button in the Run model view. A pop-up window appears to show you the progress of this run and while this is showing, you cannot edit the model. When an initial objective value has been found, a Combined Objective is displayed in a graph in this run status pop-up window. If you want to end this run before the optimal solution is obtained, you can quit by clicking Stop . When the optimal solution has been found the pop-up window closes.
When the run is completed, you can see the results in the Explore solution view. You can also click Engine statistics or Log to see the solution chart and inspect the solver engine log files. The first tab in the Explore solution view shows the objective (or objectives if you have several) with its values and weights. The Solution tables tab provides you with the best schedule with the assignment of activities to subcontractors.
You can also download the solution tables as
If your model had any conflicting constraints, these would be shown in the Conflicts tab with the Relaxations necessary to solve the model.
In the Visualization view, click Gantt to display the solution as a Gantt chart.
Create a new scenario - different model, same data
Although you have solved the model and now have the optimal schedule for your activities with subcontractors assigned, you haven't as yet considered the precedence of activities nor the skill level data of your subcontractors in making the assignments. Scenarios enable you to analyze and compare different models and data.
First, examine a new scenario with an additional constraint:
In the model view, other objectives and constraints are offered in the Suggestions pane on the right side of the window. You can add these to your model by clicking them. To see other suggestions that are not listed on the right, start typing in the search field and press enter or the refresh button. You can then browse and add from the displayed propositions.
To add the precedence constraint to your model to ensure that there are no time lags between activities:
- Duplicate Scenario 2 and call it Scenario 3. Then close the scenario pane.
- In the Suggestions pane in the model view, type in natural language activity after preceding activities for example, in the search field and click the refresh icon.
- From the new list of suggestions, click Each house_activity starts after the end of preceding
activities to add it to your constraints.
The new precedence constraint appears in your model formulation.
- Rerun the model (scenario 3) and look at the new solution. You can compare this to the solution you obtained in scenario 2, when you solved the model without this constraint. To compare solutions, open the Open scenario pane pane and click each scenario. You can also click Gantt in the Visualization view and compare solutions displayed as Gantt charts for each scenario.
- Duplicate Scenario 3 and call it Scenario 4. Then close the scenario pane.
To maximize the subcontractors' skill levels in their assignment to activities:
- In the model view, type overall
quality in the suggestions search field to find and add the following objective to your
Maximize overall quality of house_subcontractor to house_activity assignments according to table of assignment values.
Click the underlined <table of assignment value> and type or select house_expertise.
Your new objective is now Maximize overall quality of scheduling assignments according to house_expertise. Expand the objective and select Activity for the task, Subcontractor for the resource, and Skill level for the value, (table columns) to complete the definition.
You now have two objectives. You can decide whether the objectives are to be considered equally or with different weightings. You can increase and decrease the weights on each objective by using the adjacent slider. Leave the two sliders at 5 so that your two objectives are equally weighted. You can also add scale factors for the objectives. For this example, leave the scale factors as 1. For more information see Weights and scale factors displayed in the solution.
- To ensure that subcontractors only undertake tasks that they are permitted to do, add a new constraint. Type in the suggestions, for example, subcontractor must be one of possible subcontractors. You can also set Display by category to on (a tick is displayed on the switch) and select the filter Assignment to see suggestions related to assignment.
- From the filtered suggestions, find and add the following constraint: For each house_subcontractor to house_activity assignment, assigned house_subcontractors must be one of Possible Subcontractors of house_activity.
- Run the model
The new solution now shows a new assignment of subcontractors.
More about the model view
The Suggestions filter
You can also filter the suggestions to find objectives and constraints. Set Display by category to on (a tick is displayed on the switch) which opens a pane for you to select various categories of interest and apply filters to the list of suggestions. The filters enable you to see fewer suggestions. If you click the question mark icon next to the search field, you can see all possible expressions for the scheduling domain including those that are disabled. Hovering over the information icon for each expression provides you with a description. For disabled terms hovering over the expression itself also gives you an explanation for why it is disabled for this model.
The Settings tab
The Settings tab in the model view lists different scheduling and optimization parameters that can be edited. In this example the default duration unit, the optimization run time and the date/time format are shown. You can specify here a customized date/time format to suit your data.
The Data Schema tab
The Data Schema tab view lists, table by table, all information that the Modeling Assistant has imported and deduced from the input data that is necessary for the scheduling problem to be solved. You can edit certain entries in the schema which will update your model and prompt you to accept the implied model changes or cancel your edits. This can be useful for expert users for data debugging purposes. For example if a column containing an ID has been deduced as numerical, it might be useful to change this to nominal so that it can be used as a primary key.
Generating a Python notebook from your scenario
- If the scenario pane is not open, click the Scenarios icon.
- Click the three dots next to one of your scenarios and select Generate notebook.
- Enter a name for your notebook and click Generate.
You can view summary information for all your scenarios at a glance in the Overview pane. By selecting a scenario and clicking the three dots, you can perform actions such as duplicate, rename, generate a Pythonnotebook, export the scenario, or save it for deployment, for any selected scenario in this view. See Overview for more information on how to configure this pane.
In the Visualization view you can customize what you want to see displayed from any scenario. You can view your input data, your solution and add notes. For example, for this house tutorial, you can see a Gantt chart for the optimal solution schedule.
You can use table widgets and chart widgets to customize the layout of these views. You can add
headers, change background colors and other properties of your notes, tables or charts. You can
choose different types of charts such as line charts, bar charts, and so on. You can define how data
is aggregated in these charts and use the
calculate property to define how to
represent certain data values in your charts.
If you select a table or chart widget, a default instance is displayed using some of your input
data. To change the content and format of this object, click the pencil icon and edit the widget
with either the graphical editor or by editing the
You can use this view to visually compare scenarios.
See Visualization view for more information on this view's components.