4 - Tabular Data Question Review

How many files comprise this dataset?

In the Details tab of the Dataset Application, you'll find a Files field that lists the number of Dataset files and the Total number of files, which is typically the number of database files + any Spark log files generated during the transaction.

The answer may vary depending on how Spark ran the transform, but the number is likely between 5 and 8. This is relevant for Data Analysts because the performance of applications like Contour and Code Workbook that use tabular data depends on how efficiently Spark has generated these dataset files (i.e., the size and quantity of files). For more information on how Spark works, review Spark concepts or consider watching the recorded Spark Optimization training ↗.

When was the last successful transaction that assembled these files into a coherent dataset?

In the History tab, you can view all attempted builds of this dataset. In your list, you may encounter jobs that succeeded, failed, did not run, etc. The fastest way to see the date, status, and type of transaction for the current dataset view is to open the Details tab and click the Current transaction view option on the left side of the screen.

In the image below (which may vary depending on your enrollment), the latest successful transaction was a snapshot on October 6, 2022 at 9:04 AM.

What transaction type created the current view?

See answer above.

What is the most common destination for flights?

Before taking a dataset into one of Foundry's analytics applications, you can use the preview table to understand the data structure and to quickly explore dataset values. The analyst in the image below searched for a destination column and toggled the data view to by desc. count so he or she could see the destinations ordered by descending count.

Reviewing the histogram, we can see that ATL has the most flights.