Research Methods for Empirical Computer Science

CMPSCI 691DD • Spring 2009 • Mon and Wed 2:05-3:20 • CMPS 140

DescriptionScheduleProjectReviewing ReportsResourcesSubmission System

January | February | March | April | May

To access some of these readings, you will need either on-campus access,
or an OIT login to access them through the UMass library.

Date/Topic Reading

Mon Jan 26
Taking this course

[slides]

Watts, D. and S. Strogatz (1998). Collective dynamics of 'small world' networks. Nature 393: 440-442. [PDF]

Wed Jan 28
(Class canceled due to snow)

(none)

Mon Feb 02
Doing computer science as science

[slides]

Tichy, W. (1998). Should computer scientists experiment more? IEEE Computer. May. 32-40. [PDF][Alt link]

Denning, P. (2005). Is computer science science? Communications of the ACM. April. [PDF]

Wed Feb 04
Selecting research projects

[slides]

Loehle, C. (1990). A guide to increasing creativity in research - inspiration or perspiration? Bioscience 40:123-9. [PDF]

Mon Feb 09
The varieties of scientific experience

[slides]

John, G. (1995). Robust decision trees: Removing outliers from databases. Knowledge Discovery and Data Mining. 174-179. [CiteSeer]

Wed Feb 11
Why Science is Hard

[slides]

Tversky, A. and D. Kahneman (1974). Judgment under uncertainty: Heuristics and biases. Science 185:1124 - 1131.[PDF]

Mon Feb 16
(No class due to President's Day)

(none)

Wed Feb 18
Causality and Abduction

[slides]

Oates, T. and D. Jensen (1997). The effects of training set size on decision tree complexity. In Proceedings of The Fourteenth International Conference on Machine Learning. 254-262. [PS]

Optional:

Oates, T. and D. Jensen (1999). Toward a theoretical understanding of why and when decision tree pruning algorithms fail. In Proceedings of the Sixteenth National Conference on Artificial Intelligence. 372-378. [PDF]

Mon Feb 23
Hypotheses and Falsifiability

[slides]

Platt, J.R. (1964). Strong inference. Science 146:347-353. [PDF]

Optional:

Chamberlain, T.C. (1965). The method of multiple working hypotheses. Science 148:754-759. Reprint from Science February 7, 1890. [PDF]

Wed Feb 25
Strong inference (laboratory)

[slides]

(none)

Mon Mar 02
(Class canceled due to snow)

M. Liberatore, B. N. Levine, C. Barakat and C. Diot (2006). Maximizing transfer opportunities in Bluetooth DTNs.  Proc. ACM Conference on Future Networking Technologies (CoNext). [PDF]

Wed Mar 04
Exploratory data analysis

[slides]

Medawar. P. B. (1963). Is the scientific paper a fraud? In P. B. Medawar. The Threat and the Glory. New York: Harper Collins. 228-233. [PDF]

Mon Mar 09
The agony and ecstasy of research

[slides]

M. Hertz and E. Berger (2005). Quantifying the performance of garbage collection vs. explicit memory management. Proceedings of the 20th annual ACM SIGPLAN Conference on Object Oriented Programming, Systems, Languages, and Applications. 313-326.[CiteSeer]

Wed Mar 11
Personal productivity

[slides]

Richard Hamming (1986). You and Your Research. [PDF]

Mon Mar 16
Spring Break

(none)

Wed Mar 18
Spring Break

(none)

Mon Mar 23
Planning and proposing research

[slides]

L. Mackert and G. Lohman (1986). R* optimizer validation and performance evaluation for distributed queries. VLDB. 149-159. [PDF]

Wed Mar 25
Tactical decisionmaking in research
[slides]

Sco tt Kirkpatrick and Bart Selman (1994). Critical behavior in the satisfiability of random Boolean expressions. Science 264:1297-1301. [PDF]

Mon Mar 30
Tactics of experimental design I
[slides]

Domingos, P. and M. Pazzani (1997). On the optimality of the simple Bayesian classifier. Machine Learning 29:103-130. [PDF]

Wed Apr 1
No Class (instructor out of town)

(none)

Mon Apr 06
Tactics of experimental design II
[slides]

Ygge, F., and H. Akkermans (1999). Decentralized markets versus central control: A comparative study. Journal of Artificial Intelligence Research 11:301-333. [PDF]

Wed Apr 08
Experimental design (laboratory)
[slides]

D. Krioukov, K. Fall, and X. Yang (2004). Compact routing on Internet-like graphs. INFOCOM. [PDF]

Mon Apr 13
Statistical hypothesis testing

[slides]

Backstrom, L., D. Huttenlocher, J. Kleinberg, X. Lan (2006). Group formation in large social networks: Membership, growth, and evolution. Proc. 12th ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining. [PDF]

Wed Apr 15
No Class (instructor out of town)

(none)

Mon Apr 20
No Class (Patriot's Day)

(none)

Tue Apr 21
No Class (instructor out of town)

(none)

Wed Apr 22
Hypothesis testing (laboratory)

[slides]

Jensen, D., and P. Cohen (2000). Multiple comparisons in induction algorithms. Machine Learning 38(3): 309-338. [PDF]

Mon Apr 27
Modeling and parameter estimation

[modeling-slides] [writing-slides]

TBA

Wed Apr 29
No Class (instructor out of town)

(none)

Mon May 04
Strategies for studying complex systems

S. Blackburn et al. (2008). Wake up and smell the coffee: Evaluation methodology for the 21st Century. CACM. 51(8):83-89. [PDF]

Wed May 06
(Optional topics)

(none)

Mon May 11
Course wrapup

[slides]

(none)