CSC581 Homepage

Instructor:

Description:

This course is an introduction to machine learning and SVMs. We begin by framing the notion of machine learning and then develop basic concepts such as hyperplanes, features spaces and kernels necessary for the construction of SVMs. Once the theoretical groundwork has been laid we look at practical examples where this class of algorithms can been applied. Here we use machine learning as a knowledge discovery tool. We will use the statistical computing environment R for our experiments.

The goals of this course are for you,

• To have a basic understanding of machine learning and knowledge discovery.
• To be familiar with the mathematical framework of describing data and constructing models.
• To be able to apply machine learning packages in R to real-world problems.

Announcements:

Documents of Interest:

• Syllabus
• Lecture Notes
• Solutions to Assignments
• The R system
• An Introduction to R
• SVMs: Hype or Halleluja? (an alternative way of looking at SVMS)
• The Backprop Algorithm for ANNs
• Regression Trees

Assignments:

• Assignment #1: Read Chapter 1, Read Appendix B. Do problems 1.1, 1.2, and 1.4. For problem 1.4 write your program in R and demonstrate that your program works with at least two different data sets.
• Assignment #2: Read Chapters 2 through 5. Do problem 3.2 (prove the identities in Table 3.3), problem 3.4, and problem 5.3. To test your code you can use this dataset.
• Assignment #3: Read Chapter 6. Do problem 6.2. You can reuse your code from assignment #2 for parts 1 and 2 (or the code given on the solutions page). Use the QP solver given in file solve-QP.r.
• Assignment #4: Read Chapter 7. Do problems 7.1, 7.3, 7.5.
• Assignment #5: Midterm Data Proposal.
  
• Assignment #6: Do Problem 11.2. Use the svm function as implemented in e1071, but only use it in binary mode. That is, you are supposed to implement the multi-class framework around the binary svm implementation. For part b compare your multi-class implementation for the R svm to the multi-class implementation of the e1071 package usign the iris data set.
• Assignment #7: Read the The Backprop Algorithm for ANNs
. Install the 'neuralnet' package in R. Transform the iris data set into a 3-class numeric data set appropriate for training MLPs. Experiment with building MLPs. Build an MLP with the smallest **training classification error**. Hint: the plot function and the model itself report the mse network error on the training set. Can you get the classification error down to 0? What is the relationship between the MSE and the classification error? You can vary the number of hidden units to give the network different learning abilities. Write a brief report describing your findings. . Here is some example code to get you going:

# load our data set
data(iris)
# make sure the ANN library is available
library(neuralnet)
# convert the labels into numeric labels and put them into a data frame
Species.numeric <- as.numeric(iris$Species)
iris.df <- data.frame(iris,Species.numeric)
# train a neural network with two hidden nodes
net <- neuralnet(Species.numeric ~ Sepal.Width+Sepal.Length+Petal.Width+Petal.Length,iris.df,hidden=2)
# display the ANN
plot(net)
# report the ANN
net
# the training predictions from the ANN are numeric values, turn them into labels by rounding
result <- round(net$net.result[[1]])
# plot the confusion matrix
table(iris.df$Species.numeric,result)

• Assignment #8: Implement the ID3 algorithm as given in class and test it on the Play Tennis data set. Your report should include: the confusion matrix showing your classification of the training data; a copy of the tree you generated for the data set, and your source code. Extra Credit: Extend the basic ID3 algorithm with continuous variable support as discussed in class and test it on this data set. Show your generated tree and the confusion matrix.