Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running PyGWalker in a Hugging Face space #240

Open
severo opened this issue Sep 25, 2023 · 15 comments
Open

Running PyGWalker in a Hugging Face space #240

severo opened this issue Sep 25, 2023 · 15 comments
Labels
enhancement New feature or request

Comments

@severo
Copy link

severo commented Sep 25, 2023

It would be amazing to test PyGWalker in a Hugging Face space, particularly on datasets hosted on the Hub.

@longxiaofei
Copy link
Contributor

Yes! I create new space base streamlit, quickly paste pygwalker demo to this space. Then, I released a higher performance streamlit server, I haven't tried import dataset on hub yet.

https://huggingface.co/spaces/Libra-1995/test-pyg

@ObservedObserver
Copy link
Member

ObservedObserver commented Sep 25, 2023

It would be amazing to test PyGWalker in a Hugging Face space, particularly on datasets hosted on the Hub.

That's a great idea. Besides running a pygwalker app for a fixed dataset.

I am curious about is there a more native or integrated way to embed a pygwalker app for exploring any datasets in hugging face.

My ideal ways of using pygwalker with huggingface are:

  1. You can open it directly when viewing the dataset, like a place in the dataset viewer or a tab.
    image

  2. Using hugging face API to allow users to start a pygwalker workspace with any dataset in hugging face directly. The workspace can be native embedded in hugging face, a online co-lab, steamlit app, or kanaries hosting app.

@severo
Copy link
Author

severo commented Sep 26, 2023

nice @longxiaofei! Possibly we could select the dataset (plus config and split) we want to visualize, using controls. As proposed by @ObservedObserver (option 2), you can get the list of datasets from the HF API, and if you want to dig into more details (configs, splits and more), you have the Datasets Server API.

I don't know Streamlit but maybe it's easy to do with Gradio. @freddyaboulton @abidlabs do you have examples of such spaces?


@ObservedObserver on your point 1: it's a very good idea. It requires integration with the Hub, so it's more long-term, but we will think about it. It's something we have been asked for already.

@julien-c
Copy link

What about adding a badge like https://shields.io/ to dataset cards that opens a Space with the corresponding dataset_name as a query parameter? Would that work?

@longxiaofei
Copy link
Contributor

good ideas, I'll explore Hugging Face space and Datasets Server API in the next two days.

@abidlabs
Copy link

Nice! In terms of using Gradio with Pygwalker, we can embed any sort of HTML using the HTML component. So you could do for example:

with gr.Blocks() as demo:
   gr.HTML("<iframe src=...></iframe>")
demo.launch()

We're also about to release a way to create custom Gradio components, which would be interesting to explore with Pygwalker. Let me know if there's interest in exploring more.

@ObservedObserver
Copy link
Member

ObservedObserver commented Sep 27, 2023

Nice! In terms of using Gradio with Pygwalker, we can embed any sort of HTML using the HTML component. So you could do for example:

with gr.Blocks() as demo:
   gr.HTML("<iframe src=...></iframe>")
demo.launch()

We're also about to release a way to create custom Gradio components, which would be interesting to explore with Pygwalker. Let me know if there's interest in exploring more.

Yeah, we are pretty interested in exploring it. For the code above, it looks similar to Streamlit, I suppose it will work fine since pygwalker works there.
One problem that might be worried about is that pygwalker is not just a piece of HTML, it contains computation services in python kernel, which means it requires the framework to provide a communication system between client and backend.

@longxiaofei
Copy link
Contributor

What about adding a badge like https://shields.io/ to dataset cards that opens a Space with the corresponding dataset_name as a query parameter? Would that work?

I will quickly develop a POC.

@longxiaofei
Copy link
Contributor

@julien-c, kanaries can provide server support.

like this? a simple demo:

Static Badge

@ObservedObserver ObservedObserver added the enhancement New feature or request label Oct 2, 2023
@ObservedObserver ObservedObserver pinned this issue Oct 7, 2023
@ObservedObserver
Copy link
Member

Nice! In terms of using Gradio with Pygwalker, we can embed any sort of HTML using the HTML component. So you could do for example:

with gr.Blocks() as demo:
   gr.HTML("<iframe src=...></iframe>")
demo.launch()

We're also about to release a way to create custom Gradio components, which would be interesting to explore with Pygwalker. Let me know if there's interest in exploring more.

I just wrote a tutorial of how to use pygwalker in gradio, it's pretty simple for the basic usage. https://docs.kanaries.net/pygwalker/tutorials/use-pygwalker-in-gradio

But pygwalker has a advance feature that allows users to enable a duckdb engine in python kernel(which can be much faster for computations). I am wondering if it is possible gradio can support the communication between the front-end and back-end to enable this feature in pygwalker.

@abidlabs
Copy link

Awesome! As I mentioned beofre, we're working on making it possible for Gradio users to create their own custom components -- meaning that you'll be able to take an existing Gradio component and clone it, modify the backend or the frontend, and use it in your Gradio apps. If you're interested, I can share with you the current instructions for making a custom component and you could give it a shot. What do you think?

@ObservedObserver
Copy link
Member

Awesome! As I mentioned beofre, we're working on making it possible for Gradio users to create their own custom components -- meaning that you'll be able to take an existing Gradio component and clone it, modify the backend or the frontend, and use it in your Gradio apps. If you're interested, I can share with you the current instructions for making a custom component and you could give it a shot. What do you think?

That's great, would love to have a try!

@abidlabs
Copy link

Ok awesome! Here's instructions that we put together: https://github.com/gradio-app/gradio/wiki/%F0%9F%8E%A8-How-to-Make-a-Gradio-Custom-Component

Please let me know if you have any questions, happy to help!

@longxiaofei
Copy link
Contributor

longxiaofei commented Oct 25, 2023

@abidlabs Hi, Thank you very much for your support.

I have an idea to quickly connect Pygwalker to Gradio.

Most of pygwalker's communication can be done through simple http(request-response).

Is there any way for me to add a custom route to Fastapi of Gradio?

I try to hack it.(BTW, Why is the main thread blocked when prevent_thread_lock=False?).

import gradio as gr

def custom_router():
    return {"message": "hello world"}

with gr.Blocks() as demo:
    gr.HTML("hello world")

result = demo.launch(prevent_thread_lock=True)
result[0].add_api_route("/pyg_custom_route", custom_router, methods=["GET"])
demo.block_thread()

Or, use app_kwargs? example:

import gradio as gr

from starlette.routing import Route
from starlette.responses import JSONResponse
from starlette.requests import Request


def custom_router(_: Request):
    return JSONResponse({"message": "hello world"})


with gr.Blocks() as demo:
    gr.HTML("hello world")


result = demo.launch(app_kwargs={
    "routes": [Route("/custom_route", custom_router, methods=["GET"])]
})

Or, maybe add a feature: mount_fastapi_app, similar to mount_gradio_app, example:

...
mounted_app = Fastapi(...)

demo.launch(
    mounted_fastapi_app=mounted_app
)
...

def launch(...):
    ...
    app = App.create_app(blocks, app_kwargs=app_kwargs)
    if mounted_fastapi_app:
        app.mount("xxx", mounted_fastapi_app)

Maybe there are other ways? Which method do you recommend?

Looking forward to your reply.

@freddyaboulton
Copy link

Hi @longxiaofei ! Are you trying to create a custom component for PyGWalker?

If you want to send http requests from your front-end directly to your component backend, you can use the @server decorator to expose a method of your component class as an http route that your front-end code can call.

We have this working in the FileExplorer component

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

6 participants