What's the difference for the projection being Some(vec![]) and None?

[niebayes]

I found that TableProvider trait defines an interface scan that receives an Option<&Vec<usize>> as the projection. What's the difference for the projection being Some(vec![]) and None?
By the way, there's a function datafusion_common::project_schema that would return a full schema if the projection is None while return an empty schema if the projection is Some(vec![]).
So, should I follow datafusion_common::project_schema to process the projection in a consistent way as datafusion does?

[jonahgao]

None means that it returns all columns of the table; in other words, no projection has been pushed down.

vec![] means it returns no columns and provides only the number of rows, which is typically the case like count(*).

> explain select count(*) from test;
+---------------+---------------------------------------------------------------+
| plan_type     | plan                                                          |
+---------------+---------------------------------------------------------------+
| logical_plan  | Aggregate: groupBy=[[]], aggr=[[count(Int64(1)) AS count(*)]] |
|               |   TableScan: test projection=[]                               |

So, should I follow datafusion_common::project_schema to process the projection in a consistent way as datafusion does?

Yes. One example is the MemoryTable

datafusion/datafusion/physical-plan/src/memory.rs

Lines 331 to 335 in 85f92ef

	// return just the columns requested
	let batch = match self.projection.as_ref() {
	Some(columns) => batch.project(columns)?,
	None => batch.clone(),
	};

[niebayes]

@niebayes Very appreciate for your reply! I actually discovered this issue while testing SELECT count(*).