[SPARK-44860][SQL] Add SESSION_USER function #42549

vitaliili-db · 2023-08-17T23:25:10Z

What changes were proposed in this pull request?

This change implements SESSION_USER expression. It behaves exactly the same as CURRENT_USER but according to standard when respective function is used inside a routine (UDF):

CURRENT_USER should return security definer, i.e. owner of an UDF
SESSION_USER should return connected user.

The code is duplicated for this reason - to be able to identify which expression is used inside a routing.

Why are the changes needed?

This is a missing expression defined by SQL standard.

Does this PR introduce any user-facing change?

Yes, this change introduces a new expression.

How was this patch tested?

Updating existing unit tests.

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala

dtenedor

Thanks for implementing this! Generally LGTM, just one place to add more comments.

dtenedor · 2023-08-18T18:43:03Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala

@@ -302,6 +302,24 @@ case class CurrentUser() extends LeafExpression with Unevaluable {
  final override val nodePatterns: Seq[TreePattern] = Seq(CURRENT_LIKE)
 }

+// scalastyle:off line.size.limit
+@ExpressionDescription(
+  usage = """_FUNC_() - connected user name.""",


Both this comment and the existing comment for current_user on L289 above could use some more information, especially now that we have both current_user and session_user as separate functions. Even if this PR implements both functions as the same implementation, could we at least (1) dig into how the current_user function is currently implemented and describe all the cases there, and (2) mention for this session_user function that this should always refer to the identity of the invoker?

converting to alias since session_user change is not relevent for non-external functions.

yaooqinn · 2023-08-20T03:06:00Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala

+  since = "3.5.0",
+  group = "misc_funcs")
+// scalastyle:on line.size.limit
+case class SessionUser() extends LeafExpression with Unevaluable {


why not register as an alias with CurrentUser

Done, thank you!

…_user

vitaliili-db · 2023-08-21T22:33:05Z

@yaooqinn please take a look. Thank you!

yaooqinn · 2023-08-23T03:30:00Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ColumnResolutionHelper.scala

@@ -95,12 +95,13 @@ trait ColumnResolutionHelper extends Logging {
    }
  }

-    // support CURRENT_DATE, CURRENT_TIMESTAMP, and grouping__id
+  // support CURRENT_DATE, CURRENT_TIMESTAMP, CURRENT_USER, SESSION_USER and grouping__id


Suggested change

// support CURRENT_DATE, CURRENT_TIMESTAMP, CURRENT_USER, SESSION_USER and grouping__id

// support CURRENT_DATE, CURRENT_TIMESTAMP, CURRENT_USER, USER, SESSION_USER and grouping__id

yaooqinn · 2023-08-23T03:31:22Z

sql/core/src/test/scala/org/apache/spark/sql/MiscFunctionsSuite.scala

@@ -48,15 +48,16 @@ class MiscFunctionsSuite extends QueryTest with SharedSparkSession {
    checkAnswer(df.selectExpr("version()"), df.select(version()))
  }

-  test("SPARK-21957: get current_user in normal spark apps") {
+  test("SPARK-21957, SPARK-44860: get current_user, session_user in normal spark apps") {


Can we also update SPARK-21957: get current_user through thrift server?

yaooqinn

LGTM, only two minor revisons

zhengruifeng · 2023-08-24T01:22:21Z

you may need to add it the this list, since it is not supported in Python for now

vitaliili-db · 2023-08-25T18:56:32Z

you may need to add it the this list, since it is not supported in Python for now

Thank you!

yaooqinn · 2023-08-29T02:41:44Z

thanks @vitaliili-db, merged to master

### What changes were proposed in this pull request? `session_user` function was added in Scala in #42549, this PR adds it to Python ### Why are the changes needed? for parity ### Does this PR introduce _any_ user-facing change? yes ``` >>> import pyspark.sql.functions as sf >>> spark.range(1).select(sf.session_user()).show() # doctest: +SKIP +--------------+ |current_user()| +--------------+ | ruifeng.zheng| +--------------+ ``` ### How was this patch tested? ci ### Was this patch authored or co-authored using generative AI tooling? no Closes #44205 from zhengruifeng/connect_session_user. Authored-by: Ruifeng Zheng <[email protected]> Signed-off-by: Ruifeng Zheng <[email protected]>

### What changes were proposed in this pull request? `session_user` function was added in Scala in apache#42549, this PR adds it to Python ### Why are the changes needed? for parity ### Does this PR introduce _any_ user-facing change? yes ``` >>> import pyspark.sql.functions as sf >>> spark.range(1).select(sf.session_user()).show() # doctest: +SKIP +--------------+ |current_user()| +--------------+ | ruifeng.zheng| +--------------+ ``` ### How was this patch tested? ci ### Was this patch authored or co-authored using generative AI tooling? no Closes apache#44205 from zhengruifeng/connect_session_user. Authored-by: Ruifeng Zheng <[email protected]> Signed-off-by: Ruifeng Zheng <[email protected]>

[SPARK-44860] Add SESSION_USER function

7474177

github-actions bot added the SQL label Aug 17, 2023

change version

e7ae20e

srielau reviewed Aug 18, 2023

View reviewed changes

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala Outdated Show resolved Hide resolved

dtenedor approved these changes Aug 18, 2023

View reviewed changes

vitaliili-db added 2 commits August 18, 2023 18:26

remove from functions

5a9469a

remove function

bac7824

yaooqinn reviewed Aug 20, 2023

View reviewed changes

HyukjinKwon changed the title ~~[SPARK-44860] Add SESSION_USER function~~ [SPARK-44860][SQL] Add SESSION_USER function Aug 21, 2023

vitaliili-db closed this Aug 21, 2023

vitaliili-db reopened this Aug 21, 2023

convert to alias

4366570

github-actions bot added the CONNECT label Aug 21, 2023

vitaliili-db added 4 commits August 21, 2023 15:26

Merge branch 'master' of https://github.com/apache/spark into session…

a7cbc32

…_user

diff

a5fb8ae

remove broken import

0e37ddc

revert doc

0aca570

vitaliili-db requested a review from yaooqinn August 21, 2023 22:32

dtenedor approved these changes Aug 21, 2023

View reviewed changes

connect golden files

917d27b

yaooqinn reviewed Aug 23, 2023

View reviewed changes

yaooqinn approved these changes Aug 23, 2023

View reviewed changes

Address comments

9de0157

github-actions bot added the PYTHON label Aug 25, 2023

yaooqinn closed this in 600f62e Aug 29, 2023

zhengruifeng mentioned this pull request Dec 6, 2023

[SPARK-46284][PYTHON][CONNECT] Add session_user function to Python #44205

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SPARK-44860][SQL] Add SESSION_USER function #42549

[SPARK-44860][SQL] Add SESSION_USER function #42549

vitaliili-db commented Aug 17, 2023

dtenedor left a comment

dtenedor Aug 18, 2023

vitaliili-db Aug 21, 2023

yaooqinn Aug 20, 2023

vitaliili-db Aug 21, 2023

vitaliili-db commented Aug 21, 2023

yaooqinn Aug 23, 2023

vitaliili-db Aug 25, 2023

yaooqinn Aug 23, 2023

vitaliili-db Aug 25, 2023

yaooqinn left a comment

zhengruifeng commented Aug 24, 2023

vitaliili-db commented Aug 25, 2023

yaooqinn commented Aug 29, 2023

	// support CURRENT_DATE, CURRENT_TIMESTAMP, CURRENT_USER, SESSION_USER and grouping__id
	// support CURRENT_DATE, CURRENT_TIMESTAMP, CURRENT_USER, USER, SESSION_USER and grouping__id

[SPARK-44860][SQL] Add SESSION_USER function #42549

[SPARK-44860][SQL] Add SESSION_USER function #42549

Conversation

vitaliili-db commented Aug 17, 2023

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

dtenedor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vitaliili-db commented Aug 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yaooqinn left a comment

Choose a reason for hiding this comment

zhengruifeng commented Aug 24, 2023

vitaliili-db commented Aug 25, 2023

yaooqinn commented Aug 29, 2023