CSC581 Homepage
Topics in AI: Introduction to Machine Learning with Support Vector Machines
Spring 2007
Description:
Support vector machines (SVMs) belong to a new class of machine learning
algorithms with their origins firmly rooted in statistical learning
theory. Due to the strong theoretical foundation these algorithms
possess desirable properties such as the ability to learn from very
small sample sets and a firm estimation of the generalization capacity
of the learned model. These properties make this new class of learning
algorithms extremely attractive to the practitioner who is frequently
faced with "not enough data" and needs to understand "how good a
constructed model" actually is. The fact that SVMs have surpassed the
performance of artificial neural networks in many areas such as text
categorization and speech recognition bears witness to the power of this
new class of learning algorithms.
This course is an introduction to machine learning and SVMs. We begin
by framing the notion of machine learning and then develop basic
concepts such as hyperplanes, features spaces and kernels necessary for
the construction of SVMs. Once the theoretical groundwork has been laid
we look at practical examples where this class of algorithms can been
applied. When applied to tabulated data, machine learning can be viewed
as an area of computational statistics (as opposed to learning as part
of AI where the learning algorithm is part of a larger problem solving
system). In this course we take the computational statistics view and
apply SVMs to tabulated (real world) data. We will use the statistical
computing environment R for our experiments.
This is an exploratory, graduate level course, therefore strong class
participation is expected. Readings will be mainly assigned from a book
draft, chapters will be made available online as we move forward.
Assignments will consist of problem sets and programming assignments
using the open source R environment. For the midterm exam you are
expected to write implement a simple SVM algorithm and demonstrate that
it works. For the final examination you are expected to build an SVM
model on a given data set and write up a report analyzing your findings.
Final Projects due on May 10th @ 2pm in my office (see problem sheet below)
NOTE: Chapter 10 is coming...perhaps not...I have some rough notes, but I think
the lecture notes are much more comprehensive at this point...
Announcements:
[4/23/07] Posted the final problem sheet.
[4/23/07] Posted the paper on nu-SVMs.
[4/11/07] For those folks who are taking the CSC581 course for 4 credits I just posted
an additional assignment. This is due on 4/30 in my office.
[4/11/07] Posted assignment #8
[3/31/07] Hint for the homework: before splitting into k folds for cross-validation you
should randomize your dataset in case it is sorted by some criterion. See the midterm for
the R code.
[4/3/07] Posted Ron Kohavi's paper on cross-validation.
[3/29/07] Posted chap 9 and Platt's paper.
[3/29/07] Posted assignment #7
[3/12/07] posted the midterm
[3/9/07] posted chapter 8
[3/7/07] posted assignment #6
[3/7/07] posted the kernel-adatron paper.
[3/4/07] Posted chapter 7 on support vector machines...apologies for the delay.
[2/28/07] NOTE: **no** homework due for next week (except for the reading, chapter to follow), I decided against the dual perceptron.
[2/23/07] posted chapter 6
[2/22/07] posted assignment #5
[2/14/07] posted chapter 5
[2/12/07] Posted assignemnt #4
[2/8/07] Posted chapter 4
[2/7/07] Posted chapter 3
[2/5/07] Posted assignment #3
[2/5/07] Schedule change: class will start at 5:30pm beginning 2/6.
[1/25/07] Room change: starting immediately we will be in Washburn Rm 220
[1/22/07] Welcome!
Documents of Interest:
- Syllabus
- Lecture Notes
- Book Chapters
- The R system
- An Introduction to R
- K. Bennett and C. Campbell, Support
Vector Machines: Hype or Hallelujah? SIGKDD Explorations, 2:2,
2000, 1-13.
- T. T. Friess, N. Cristianini, and C. Campbell,
The kernel adatron algorithm: a fast and simple learning procedure for support vector machine.
In Proc. 15th International Conference on Machine Learning, Morgan Kaufman Publishers, 1998.
- J. C. Platt,
Sequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines,
Microsoft Research Technical Report MSR-TR-98-14, 1998.
- R. Kohavi, A study of cross-validation and bootstrap for
accuracy estimation and model selection,
in Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, pages 1137--1143. San Mateo, CA: Morgan Kaufmann, 1995.
- B. Schoelkopf, A. Smola, R. C. Williamson, and P. L. Bartlett. New Support Vector Algorithms.
Neural Computation, 12:1207-1245, 2000.
Data Sets:
Many of the packages above have accompanying data sets. But the
premier source for experimental machine learning data sets is the UCI
Machine Learning Repository. The Statlib library
at CMU is another great place to look for data.
Assignments:
- Assignment #1: Reading: Draft - Chap 1, Introductory Statistics - Chaps 1,2,3; Explore the built-in 'cars'
data frame and write a brief 1-2 page report on your findings. Due 1/30 in class.
- Assignment #2: Reading: Draft - Chap 2, Intro Stats - Chap 5.1; problem sheet,
Due 2/6 in class.
- Assignment #3: Reading: Draft - Chap 3 & 4; problem sheet (make sure it is verion 1.2),
Due 2/13 in class.
- Assignment #4: Reading: Draft - Chap 5; problem sheet, modified wrapper function
for convex optimizer,
Due 2/21 in class.
- Assignment #5: Reading: Draft - Chap 6; problem sheet,
Due 2/27 in class.
- Assignment #6: Reading: Draft - Chap 7,8,9; problem sheet,
Due 3/13 in class.
- Midterm: problem sheet, Due 3/27 in class.
- Assignment #7: Reading: Draft - Chap 9,10; problem sheet,
Due 4/3 in class.
- Assignment #8: problem sheet,
Due 4/17 in class.
- Final: problem sheet, Due 5/10 @ 2pm in my office.
Instructor:
Dr. Lutz Hamel
Tyler, Rm 251
Office Hours: TBA
email: lutz at inductive dash reasoning dot com