Data Mining & Knowledge Discovery

(Mineração de Dados e Descoberta de Conhecimento)

DAT003/CAIA003 - CPGEI & PPGCA

last update: 28/04/2025 22:08

General issues:

•  Lecturers: Prof. Heitor Silvério Lopes e Prof. Thiago Henrique Silva
•  Check-in survey: https://forms.gle/i8GYFjkqR897igwA9
•  Classes' modality: Hybrid
  • For in-person classes: room C301 campus Centro
  • For online classes: http://meet.google.com/nvj-omnc-kyi
•  Time: tuesdays (08:20-11:40)
•  Duration/credits: ~45 hours (12 weeks) / 3 credits
•  Grading system: grades A, B, C are approved; grades D, E are failed
•  Moodle portal: https://moodle.utfpr.edu.br/course/view.php?id=27012
•  Motivations:
  • The huge availability of data of all types stored everywhere (databases, data warehouses, bank transactions, e-commerce, web pages, social networks, etc) fostered research in a specific area of Computer Science: Knowledge Discovery in Databases (KDD). One of the main steps of KDD is the application of algoriths for Data Mining (DM). The objective of DM is the discovery of human-comprehensible patterns hidden in large amounts of data. DM & KDD are multidisciplinar, ovelapping many areas such as Databases, Computational Intelligence, Machine Learning, Pattern Recognition, Information Science and Statistics. The application of such methods are omnipresent, leading to a growing interest not only by researchers but, also, by companies and government.
•  Objective:
  • This course aims at presenting the knowledge discovery process using data mining methods. The main data mining tasks are studied along with the most well-known algorithms, focusing on real-world applications.
•  Syllabus (tentative):
  •  What is data mining. Knowledge discovery process. Data collection, preparation, cleaning and visualization. Data mining tasks: classification, association, clustering, feature selection & dimensionality reduction. Advanced topics: text mining, multimedia (image, signals, sound) mining. Application of different algorithms for the main data mining tasks, evaluation and interpretation of results.
•  Evaluation:
  •  Weekly homeworks: 30%, final project: 70%.
  • Homeworks require a short report with results which must be upload using at the Moodle portal.
  • Final project require a "paper-like" full-report with some theoretical background, motivations, objective, methods, experiments and results.
  • Oral presentations of the final projects are scheduled as a seminar.
•  Suggested literature:
  • P-N. Tan; M. Steinbach; V. Kumar. Introduction to Data Mining, 2nd edition. Pearson, 2018.
  • Witten, I.H., Frank, E. e Hall, M.A. Data Mining: Practical Machine Learning Tools and Techniques, 3rd edition. Morgan Kaufmann, 2011.
  • J. Han; M. Kamber e Pei, J. Data Mining: Concepts and Techniques. 3rd. edition, Morgan Kaufmann, 2012.

Tentative scheduling for 2025-1^st (subject to changes):

Homework:

Additional support materials and links:

•  Softwares:
  • Orange v. 3.38.1
  • Anaconda
  • Weka v. 3.9.1 (manual)
  • Gephi v. 0.10.0
  • R datamining
  • Matlab, Scilab, GNU Octave
  • Rapidminer
  • Knowledge Extraction based on Evolutionary Learning - KEEL
  • Knime
  • Natural Language Toolkit - NLTK
  • Conversor csv2arff
  • BioWeka
  • Cluster/PyCluster/PerlCluster
  • Scikit-learn
  • TextProcessingTool.jar
•  Original slides from books:
  • Book Tan, Steibach e Kumar
  • Book Witten, Frank and Hall (chapters 1-5 and 6-8)
  • Book Han, Kamber e Pei
•  Datasets:
  • Kaggle
  • Sensor information system
  • IEEE Dataport
  • UCI Machine Learning repository, mirror
  • Wikipedia: List of datasets for machine learning research
  • Google Dataset Search
  • Opendatasoft
  • Mendeley Data
  • Cancer program datasets
  • Awesome public datasets
  • Reddit datasets
  • FlowingData
  • KDD Cup
  • Macro and Micro Economic data
  • European Data Portal
  • U.S. Government's open data
  • Open data gateways:
    • Portal Brasileiro de dados abertos
    • Open data projects of Brazilian cities: Curitiba, São Paulo, Rio de Janeiro, Porto Alegre and Recife
    • U.S. Government open data
    • ICSU World Data System
  • Specific areas:
    • Regression datasets site1 e site2
    • Text mining datasets: HP e Reuters
    • Gene expression datasets: Array express e Gene Expression Omnibus
    • Time series datasets: Time Series Data Library, UCR Time Series Classification Archive
    • Graphs & Networks datasets: Stanford Large Network Dataset Collection, Network Repository

•  On-line courses:
  • Programação Genética: Fundamentos e Aplicações Reais (Youtube)
  • Data Mining with Weka (Youtube)
  • R Programming (Coursera)
•  Selected papers:
  • Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Yu, P.S., Zhou, Z-H., Steinbach, M., Hand, D.J., Steinberg, D. Top 10 algorithms in data mining. Knowledge and Information Systems, v. 14, p. 1-37, 2008.
  • Jain, D., Gautam, S. Implementation of Apriori Algorithm in Health Care Sector: a Survey. International Journal of Computer Science and Communication Engineering, v. 2, n. 4, p. 26-32, 2013.
  • Bouckaert, R.R., Frank, E., Hall, M.A., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.,  WEKA - Experiences with a Java Open-Source Project. Journal of Machine Learning Research, v. 11, p. 2533-2541, 2010.
•  Other relevant links:
  • Data visualization tools
  • OpenDatasoft
  • KDDNuggets
  • MIT MATLAB cheatsheet
  • Matrix operations in MATLAB-R-Python-Julia
  • Python-Scikit cheatsheet