Build a baseline before tuning, with explicit safeguards against data leakage.
ScienceSkills editorial guide · AI-assisted planning material, not a tested execution report.
1. Define the prediction task
Identify the target, observation unit and time at which a prediction will be made. Remove features that would only become available after that time.
2. Choose an evaluation split
Use a split appropriate to the data: by time for forecasting or by group when repeated observations share a person, device or location. Reserve the final test set before model selection.
3. Fit a transparent baseline
Fit preprocessing only on training data. Compare a simple baseline with the selected model using a metric that matches the task and class distribution. Record all parameters and seeds.
4. Inspect errors and explanations
Evaluate subgroup errors and calibration where relevant. Use explanation methods to inspect model behaviour, while keeping predictive association distinct from causality.
Expected deliverables
A versioned dataset description, split specification, fitted pipeline, evaluation report and environment record.
A starter prompt
Help me plan a tabular prediction baseline for [target] using [data description]. Ask about observation units and time leakage first. Propose a split, baseline and evaluation metrics. Do not fit preprocessing on the test data.
Confirm details in the linked skill documentation and underlying tool references before use.
Supports machine learning in Python with scikit-learn. Applies when working with supervised learning (classification, regression), unsupervised learning (clustering, dimensionality reduction), model evaluation, hyperparameter tuning, preprocessing, or building ML pipelines. Provides comprehensive reference documentation for algorithms, preprocessing techniques, pipelines, and best practices.
Explain and audit machine-learning predictions with SHAP. Use for selecting SHAP explainers and maskers, computing and validating feature attributions, handling multi-output explanations, and producing local or global SHAP visualizations.
Guided statistical analysis for research data - test selection, assumption checking, effect sizes, power analysis, Bayesian alternatives, and APA-formatted reporting. Use whenever a user wants to compare groups, test a hypothesis, analyze experimental or survey data, check statistical assumptions, compute required sample sizes, or write up results - even if they never name a specific test. Covers t-tests, ANOVA, chi-square, correlation, regression, non-parametric and Bayesian methods. For low-level model APIs, see the statsmodels and pymc skills.
ScienceSkills