The difference is that datasets are presented to you in full - all available variables - for you to manipulate and aggregate as needed. Statistics are typically presented in aggregate form already as a table, chart or graph.
When you want quick facts for a presentation or paper (typically used to strengthen a point), you would use prepared statistics. Just make sure you cite them properly. If you're planning to test a hypothesis, develop a comprehensive marketing strategy, assess the effect of a particular treatment - essentially anything that would require in-depth analyses - then you need a dataset.
When you collect data for your own research, this is considered primary data collection. When you use a dataset that has been collected by somone else, you are using what is considered secondary data.