CSC581 Homepage

Topics in AI: Introduction to Machine Learning with Support Vector Machines

Spring 2014

Instructor:

Dr. Lutz Hamel
Tyler, Rm 251
Office Hours: Tue 12:15-1:15pm, Wed 2-3pm
email: hamel@cs.uri.edu

Description:

Support vector machines (SVMs) belong to a new class of machine learning algorithms with their origins firmly rooted in statistical learning theory. Due to the strong theoretical foundation these algorithms possess desirable properties such as the ability to learn from very small sample sets and a firm estimation of the generalization capacity of the learned model. These properties make this new class of learning algorithms extremely attractive to the practitioner who is frequently faced with "not enough data" and needs to understand "how good a constructed model" actually is. The fact that SVMs have surpassed the performance of artificial neural networks in many areas such as text categorization, speech recognition and bioinformatics bears witness to the power of this new class of learning algorithms.

This course is an introduction to machine learning and SVMs. We begin by framing the notion of machine learning and then develop basic concepts such as hyperplanes, features spaces and kernels necessary for the construction of SVMs. Once the theoretical groundwork has been laid we look at practical examples where this class of algorithms can been applied. Here we use machine learning as a knowledge discovery tool. We will use the statistical computing environment R for our experiments.

The goals of this course are for you,

Announcements:

** Final due Friday 5/9 @ noon in my office **

[4/30/14] Solutions page is now up to date
[4/16/14] posted assignment #7
[4/9/14] posted assignment #6
[3/26/14] ** posted the midterm due Thursday 4/3 in class **
[3/19/14] posted solutions for #4
[3/12/14] posted solution #3
[2/27/14] ** no class Tuesday 3/4 **
[2/27/14] posted assignment #4 and #5
[2/20/14] posted solutions to assignment #2
[2/20/14] posted assignment #3
[2/10/14] posted assignment #2.
[2/3/14] More hints for problem 1.4. Here is a learner that constructs a model that given an appropriate object will always return the first label found in the training data set.
# The file contains a function 'learner' that constructs
# a model function  based on the first label of the
# dependent attribute in the training data that it finds.
# The learner makes the assumption that the
# dependent attribute is always the last column
# in the training data.
#
# use: 
#  model <- learner() # to build the model
#  model(x) # to make some prediction of object x

learner <- function(training.df) {

	# make sure we are actually handed a data frame
	if (!is.data.frame(training.df))
	   stop("not a data frame")
	
	# find the number of columns
	n <- ncol(training.df)
    
    # find the first label in the training data
	label <- training.df[[1,n]]

    # build our model
    # our model always returns the label that appeared first in the
    # training data
    function(x) label
}
Assume that the above code was saved in the file 'first-label.r' in some directory.
### get the current working directory
> getwd()
[1] "/Users/lutz/Documents/Courses/2014/spring2014/csc581/handouts/solutions/assignment #1"
### my function is saved in the 'code' subfolder
> setwd("code")
### read in the function definition
> source("first-label.r")
### let's make sure the learner is what we expect it to be...
> learner
function(training.df) {

	# make sure we are actually handed a data frame
	if (!is.data.frame(training.df))
	   stop("not a data frame")
	
	# find the number of columns
	n <- ncol(training.df)
    
    # find the first label in the training data
	label <- training.df[[1,n]]

    # build our model
    # our model always returns the label that appeared first in the
    # training data
    function(x) label
}
### looks good ... load a data set
> data(iris)
### build a model
> m <- learner(iris)
### let's take a look at the model -- the model consists of a function with an appropriate
### environment that has the variable 'label' defined.
> m
function(x) label
environment: 0x100d0cdb8
### build a data frame that only has object descriptions, no labels
> objects <- subset(iris,select=-Species)
### apply the model to the first object ... note the notation!!
> m(object[[1,]])
[1] setosa
Levels: setosa versicolor virginica
### apply the model to the 100th object
> m(object[[100,]])
[1] setosa
Levels: setosa versicolor virginica
[1/30/14] Here are some additional hints for assignment #1, assume that training.df is a data frame,
	n <- ncol(training.df) # number of columns in a frame
	target.attribute <- training.df[[n]] # another way of retrieving columns from a frame using the [[ ]] notation
	target.levels <- table(target.attribute) # tabulate the levels in the target attribute
	ix <- which.max(target.levels) # find out which level appears most often
	majority.label <- names(target.levels[ix]) # convert the level descriptor into a string
[1/28/14] Posted assignemnt #1.
[1/16/14] Welcome!

Documents of Interest:

Data Sets:

Many of the packages above have accompanying data sets.  But the premier source for experimental machine learning data sets is the UCI  Machine Learning Repository. The Statlib library at CMU is another great place to look for data.

Assignments: