mirror of
https://github.com/bvanroll/college-datascience.git
synced 2025-08-29 12:02:45 +00:00
260 lines
16 KiB
Plaintext
260 lines
16 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Labo 7 Data Science: Decision Trees"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"<div class=\"alert alert-block alert-warning\">\n",
|
|
"<strong>Opmerking.</strong> In dit labo worden decision trees getekend a.d.h.v. het pakket GraphViz. In labo 4 & 5 werd dit pakket geïnstalleerd. Op je eigen PC installeer je de package <strong>python-graphviz</strong> in Anaconda (onder environments). Dit kan een tijdje duren. Herstart je kernel om de installatie te kunnen gebruiken.\n",
|
|
"</div>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Oefening 1** : **Decision Trees voor een eenvoudig classificatieprobleem**\n",
|
|
"\n",
|
|
"**1.1 De data verkennen**\n",
|
|
"\n",
|
|
"Gegeven de dataset van housing-ny-sf.csv. Deze dataset kan worden gebruikt om te voorspellen of een appartement in New York gelegen is of in San Fransisco. Het bestand bevat volgende kolommen:\n",
|
|
" * in_sf: het te voorspellen target: staat op 1 indien het appartement in San Francisco gelegen is\n",
|
|
" * beds: het aantal bedden\n",
|
|
" * bath: het aantal baden\n",
|
|
" * price: de verkoopprijs (\\$)\n",
|
|
" * year_built: het bouwjaar\n",
|
|
" * sqft: de oppervlakte in square foot\n",
|
|
" * price_per_sqft: de prijs (\\$) per square foot\n",
|
|
" * elevation: hoogte in m\n",
|
|
"\n",
|
|
"Een leuke visuele intro op deze oefening vind je hier: _http://www.r2d3.us/visual-intro-to-machine-learning-part-1/_\n",
|
|
"\n",
|
|
" * Laad de data in in een Pandas-dataframe (gelieve niks te veranderen aan het csv-bestand, tip: skippen).\n",
|
|
" * Maak een scatter_matrix-plot van de __features__ waarbij elke instanties steeds ingekleurd wordt volgens zijn target (met colormap 'brg' wordt San Francisco groen en New York blauw)\n",
|
|
" * Teken met Pandas (groupby en hist(alpha=0.4)) een histogram (met verschillende kleur voor SF en NY) voor een aantal features waarvan je verwacht dat de spreiding voor de 2 steden sterk verschilt"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [
|
|
{
|
|
"ename": "ParserError",
|
|
"evalue": "Error tokenizing data. C error: Expected 3 fields in line 3, saw 8\n",
|
|
"output_type": "error",
|
|
"traceback": [
|
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
|
|
"\u001b[0;31mParserError\u001b[0m Traceback (most recent call last)",
|
|
"\u001b[0;32m<ipython-input-7-d1e7aebbc63b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;32mfrom\u001b[0m \u001b[0msklearn\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mtree\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;31m#dees moe gewoon gefixed worde, geen id hoe, ma fuck it dude\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0mhousing\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"housing-ny-sf.csv\"\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mskip_blank_lines\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mskipinitialspace\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/.local/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mparser_f\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)\u001b[0m\n\u001b[1;32m 700\u001b[0m skip_blank_lines=skip_blank_lines)\n\u001b[1;32m 701\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 702\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 703\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 704\u001b[0m \u001b[0mparser_f\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__name__\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/.local/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m 433\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 434\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 435\u001b[0;31m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mparser\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnrows\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 436\u001b[0m \u001b[0;32mfinally\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 437\u001b[0m \u001b[0mparser\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/.local/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mread\u001b[0;34m(self, nrows)\u001b[0m\n\u001b[1;32m 1137\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnrows\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1138\u001b[0m \u001b[0mnrows\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_validate_integer\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'nrows'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnrows\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1139\u001b[0;31m \u001b[0mret\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnrows\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1140\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1141\u001b[0m \u001b[0;31m# May alter columns / col_dict\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32m~/.local/lib/python3.7/site-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mread\u001b[0;34m(self, nrows)\u001b[0m\n\u001b[1;32m 1993\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnrows\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1994\u001b[0m \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1995\u001b[0;31m \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnrows\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 1996\u001b[0m \u001b[0;32mexcept\u001b[0m \u001b[0mStopIteration\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 1997\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_first_chunk\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
|
|
"\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader.read\u001b[0;34m()\u001b[0m\n",
|
|
"\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader._read_low_memory\u001b[0;34m()\u001b[0m\n",
|
|
"\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader._read_rows\u001b[0;34m()\u001b[0m\n",
|
|
"\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader._tokenize_rows\u001b[0;34m()\u001b[0m\n",
|
|
"\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.raise_parser_error\u001b[0;34m()\u001b[0m\n",
|
|
"\u001b[0;31mParserError\u001b[0m: Error tokenizing data. C error: Expected 3 fields in line 3, saw 8\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import pandas as pd\n",
|
|
"from sklearn.tree import DecisionTreeClassifier\n",
|
|
"from sklearn.model_selection import train_test_split\n",
|
|
"from sklearn import tree\n",
|
|
"#dees moe gewoon gefixed worde, geen id hoe, ma fuck it dude\n",
|
|
"housing = pd.read_csv(\"housing-ny-sf.csv\",skip_blank_lines=True,skipinitialspace=True)\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"\n",
|
|
"train_test_split(housing.iloc[:,1:], housing['in_sf'], test_size=0.3, random_states=0)\n",
|
|
"\n",
|
|
"#housing.groupby('in_sf').\n",
|
|
"housing.groupby('in_sf').price.hist(alpha=0.4)\n",
|
|
"\n",
|
|
"for i in range(1,10):\n",
|
|
" clf - tree.DecisionTreeClassifier(random_states=0, max_depth=i)\n",
|
|
" clf = clf.fit(x_train, y_train)\n",
|
|
" print(clf.score(x_train, y_train))\n",
|
|
" print(clf.score(x_test, y_test))\n",
|
|
" print(\"\\n/n\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"clf = tree.DecisionTreeClassifier(random_state=0,max_depth=3)\n",
|
|
"clf = clf.fit(x_train, y_train)\n",
|
|
"\n",
|
|
"import graphviz\n",
|
|
"dot_data = tree.export_graphviz(clf, out_file=Nonem feature_names=housing.iloc[:,1:].columns, class_names=['New York', 'San Fransisco'] filled=True, rounded=True, special_characters=True)\n",
|
|
"graph = graphviz.Source(dot_data)\n",
|
|
"graph"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**1.2 Training en parameter tuning**\n",
|
|
"\n",
|
|
" * Deel de data in in een trainingset en een test set (70%/30%) - kies een random_state ≠ 0 bv. 88\n",
|
|
" * Train deze data met DecisionTreeClassifier zonder parameters\n",
|
|
" * Schrijf een script dat de ideale diepte zoekt van de decision tree en teken deze tree"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**Oefening 2** : **Classificatie-oefening met decision trees, random forests en gradient boosting machines**\n",
|
|
"\n",
|
|
"**2.1 Generatie van de sample data**\n",
|
|
"\n",
|
|
"Je start deze oefening met de creatie van sample data voor een 'ternair classificatieprobleem'. Deze data heeft 2 features nl. X en Y, en een target genaamd __color__ (mogelijke waarde: red, green, blue). We zullen deze data gebruiken om met verschillende classificatie-algoritmen te testen en de decision boundary te visualiseren.\n",
|
|
"\n",
|
|
"* Door x- en y-coördinaten te genereren met _np.random.normal_ ontstaat er een _puntenwolk_ met als centrum (0,0). Door een constante waarde bij x of y te tellen kan je het centrum van deze wolk verschuiven in de x- of y-richting. Genereer nu volgende sample-data:\n",
|
|
" * een puntenwolk van 1000 instanties met als centrum (0,0) en color-label 'red'\n",
|
|
" * een puntenwolk van 1000 instanties met als centrum (2.5,2.5) en color-label 'green'\n",
|
|
" * een puntenwolk van 1000 instanties met als centrum (5,0) en color-label 'blue'\n",
|
|
" * bij de aanroep van _np.random.normal_ gef je geen extra parameters mee, tenzij de size\n",
|
|
" \n",
|
|
"* Maak een scatter-plot van deze data. Als alles goed zit, zie je de 3 apart ingekleurde puntenwolken die lichtjes overlappen. (Je kan de color-kolom rechtstreeks doorgeven aan matplotlib.) Met volgende code kan je ervoor zorgen dat X en Y dezelfde schaal hanteren:\n",
|
|
"\n",
|
|
" <code>plt.gca().set_aspect('equal', adjustable='box')</code>"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**2.2 Decision trees: visualisatie van de decision boundary**\n",
|
|
"\n",
|
|
"* Deel de data in in een training- en testset (70%/30%)\n",
|
|
"* Train een DecisionTreeClassifier met de trainingsdata en meet de accuracy op de training- en testdata\n",
|
|
"* We gaan nu de decision boundary benaderen door eerst de voorspelling op te vragen voor een grid van (x,y)-coördinaten die de volledige grafiek bedekt. Deze grid genereer je als volgt:\n",
|
|
"\n",
|
|
"<code>grid = np.mgrid[-4:8.6:0.05, -4:6:0.05].reshape(2,-1).T</code>\n",
|
|
"\n",
|
|
"* De voorspelde waarden kan je ook weer rechtstreeks doorgeven als kleur van de scatter-plot\n",
|
|
"\n",
|
|
"\n",
|
|
"* Pas nu je script aan zodat je voor max_depth van 1 t.e.m. 8 de accuracies print en de decision boundary plot\n",
|
|
"\n",
|
|
"Kan je de decision boundary van max_depth=1 verklaren? Kan je de instelling met de beste bias-variance tradeoff ook visueel verklaren a.d.h.v. de decsion boundary?"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"**2.3 Random forests en gradient boosting machines**\n",
|
|
"\n",
|
|
"* Toon nu ook de accuracies en de decision boundary voor Random forests en gradient boosting machines\n",
|
|
"* Random forests: waarom heeft parameter tuning van max_features hier geen zin?\n",
|
|
"* Gradient boosting machines: experimenteer eens met de learning_rate ."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.7.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|