First commit.
This commit is contained in:
5
.gitignore
vendored
Normal file
5
.gitignore
vendored
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
node_modules/
|
||||||
|
python/__pycache__/
|
||||||
|
models/
|
||||||
|
*.pyc
|
||||||
|
package-lock.json
|
||||||
191
INNER_WORKING.md
Normal file
191
INNER_WORKING.md
Normal file
@@ -0,0 +1,191 @@
|
|||||||
|
# Node.js Python Binding Class Documentation
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This project provides a Node.js interface (`python_bindings.js`) to interact with a Python backend. The Python backend exposes an API via a Controller class (`controller.py`) and communicates with Node.js via JSON messages over stdin/stdout using `python-shell` in Node.js.
|
||||||
|
|
||||||
|
The Node.js class `Bindings` wraps the interaction with the Python process and exposes asynchronous methods mapped dynamically to Python Controller methods. It supports property setting/getting, method calling with parameters, and streaming partial results via callbacks.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Files and Components
|
||||||
|
|
||||||
|
### 1. `index.js`
|
||||||
|
|
||||||
|
- Entry point example illustrating usage of the `ChatModel` (the default export from `python_bindings.js`).
|
||||||
|
- Shows setting properties, calling methods, receiving streamed messages, and proper shutdown.
|
||||||
|
- This file is intended for customization by users for their use case.
|
||||||
|
|
||||||
|
Example usage:
|
||||||
|
|
||||||
|
```js
|
||||||
|
import ChatModel from "./python_bindings.js";
|
||||||
|
|
||||||
|
const chat = new ChatModel({ "model": "something" });
|
||||||
|
|
||||||
|
await chat.setProperty("tokenizer", 123);
|
||||||
|
|
||||||
|
const modelResponse = await chat.getModelPath();
|
||||||
|
console.log(modelResponse); // { model: "something" }
|
||||||
|
|
||||||
|
const tokenizerResponse = await chat.getTokenizer();
|
||||||
|
console.log(tokenizerResponse); // { tokenizer: 123 }
|
||||||
|
|
||||||
|
let response = await chat.increment({ by: 5 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
chat.onMessage(function(data) {
|
||||||
|
console.log(data); // receives streamed partial results
|
||||||
|
});
|
||||||
|
|
||||||
|
chat.testStream();
|
||||||
|
|
||||||
|
chat.end();
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2. `python_bindings.js`
|
||||||
|
|
||||||
|
- Implements `Bindings` class which manages the Python subprocess using `python-shell`.
|
||||||
|
- Sends JSON-formatted method calls with parameters to Python.
|
||||||
|
- Receives JSON responses asynchronously.
|
||||||
|
- Supports streaming JSON messages prefixed with `__STREAM__`.
|
||||||
|
- Uses Proxy to dynamically route method calls to Python backend methods.
|
||||||
|
- Provides:
|
||||||
|
- `.setProperty(name, value)` to set Python controller properties.
|
||||||
|
- `.getProperty(name)` to get Python controller properties.
|
||||||
|
- `.onMessage(callback)` to register streaming data callback.
|
||||||
|
- `.end()` to gracefully end the Python subprocess.
|
||||||
|
|
||||||
|
Key methods:
|
||||||
|
|
||||||
|
- `callMethod(method, ...args)`: sends a method call to Python and returns a promise resolving to the response.
|
||||||
|
- `processSendQueue()`: ensures requests are sent sequentially.
|
||||||
|
- `onMessage(callback)`: sets a callback for streaming partial data.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 3. `python/controller.py`
|
||||||
|
|
||||||
|
- Defines `Controller` class that inherits from `BaseController`.
|
||||||
|
- Implements backend logic called by Node.js methods.
|
||||||
|
- Example methods:
|
||||||
|
- `getModelPath(params)`
|
||||||
|
- `getTokenizer(params)`
|
||||||
|
- `increment(params)`
|
||||||
|
- `reset(params)`
|
||||||
|
- `testStream(params)` — emits streaming partial data via `self.send()`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 4. `python/basecontroller.py`
|
||||||
|
|
||||||
|
- Provides base class with common controller features:
|
||||||
|
- `setProperty(params)`: sets attributes dynamically on controller.
|
||||||
|
- `getProperty(params)`: gets attributes dynamically.
|
||||||
|
- `send(data)`: sends partial streaming data through injected stream function.
|
||||||
|
- `set_stream_func(stream_func)`: used to inject the stream callback for `send()`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 5. `python/router.py`
|
||||||
|
|
||||||
|
- Acts as the Python message router.
|
||||||
|
- Reads JSON messages from `stdin`, calls the appropriate `Controller` method, and writes JSON responses to `stdout`.
|
||||||
|
- Supports streaming partial responses with the `__STREAM__` prefix.
|
||||||
|
- Handles unknown methods gracefully by returning error JSON.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 6. `python/index.py`
|
||||||
|
|
||||||
|
- Runs the `Router` instance when executed as a script.
|
||||||
|
- This is the Python entry point for the subprocess.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 7. `router.js`
|
||||||
|
|
||||||
|
- Alternative Node.js approach (not used directly by `Bindings`).
|
||||||
|
- Spawns a Python process per method call.
|
||||||
|
- Supports streaming messages.
|
||||||
|
- More suited for stateless or separate process calls.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## How It Works (Workflow)
|
||||||
|
|
||||||
|
1. **Initialization**
|
||||||
|
|
||||||
|
- `Bindings` starts a persistent Python subprocess running `python/index.py` via `python-shell`.
|
||||||
|
- The Python process reads JSON messages from stdin and sends JSON responses on stdout.
|
||||||
|
|
||||||
|
2. **Method Calls**
|
||||||
|
|
||||||
|
- Node.js sends JSON messages with `{ method: "methodName", params: { ... } }`.
|
||||||
|
- Python calls the matching method in `Controller` class with params.
|
||||||
|
- Python sends back `{ result: ... }` or `{ error: ... }`.
|
||||||
|
|
||||||
|
3. **Streaming**
|
||||||
|
|
||||||
|
- Python can send partial results during long-running methods using the injected `send()` function.
|
||||||
|
- These partial messages are prefixed with `__STREAM__` so Node.js can route them to the streaming callback.
|
||||||
|
|
||||||
|
4. **Dynamic Proxy**
|
||||||
|
|
||||||
|
- The `Bindings` class uses a JavaScript Proxy to make any method call on the object automatically send the request to Python and return a Promise with the result.
|
||||||
|
|
||||||
|
5. **Property Setting/Getting**
|
||||||
|
|
||||||
|
- Properties like `model` and `tokenizer` can be set or fetched through dedicated calls or via `.setProperty()`.
|
||||||
|
|
||||||
|
6. **Shutdown**
|
||||||
|
|
||||||
|
- Calling `.end()` ends the Python process cleanly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Usage Summary
|
||||||
|
|
||||||
|
- Import `Bindings` or `ChatModel`.
|
||||||
|
- Instantiate with initial properties if desired.
|
||||||
|
- Call any controller method asynchronously on the instance.
|
||||||
|
- Use `.onMessage()` to handle streamed data.
|
||||||
|
- Call `.end()` to terminate the backend.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes for Customization
|
||||||
|
|
||||||
|
- Edit `controller.py` to add your own Python logic and methods.
|
||||||
|
- Update `index.js` for your application-specific flow.
|
||||||
|
- Ensure Python methods return JSON serializable objects.
|
||||||
|
- Implement streaming with `send()` and handle it in Node.js via `.onMessage()`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
import ChatModel from "./python_bindings.js";
|
||||||
|
|
||||||
|
const chat = new ChatModel({ model: "gpt" });
|
||||||
|
|
||||||
|
await chat.setProperty("tokenizer", 42);
|
||||||
|
|
||||||
|
const info = await chat.getModelPath();
|
||||||
|
console.log(info); // { model: "gpt" }
|
||||||
|
|
||||||
|
chat.onMessage(data => {
|
||||||
|
console.log("Streamed partial data:", data);
|
||||||
|
});
|
||||||
|
|
||||||
|
await chat.testStream();
|
||||||
|
|
||||||
|
await chat.end();
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
This documentation provides a comprehensive understanding of the binding class design and usage to integrate Python backend logic into a Node.js environment efficiently.
|
||||||
110
README.md
110
README.md
@@ -1,2 +1,110 @@
|
|||||||
# Python-bindings-for-nodejs
|
# Python Binding for Node.js
|
||||||
|
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This project enables seamless integration of a Python backend with a Node.js application by maintaining a persistent Python subprocess. It allows calling Python methods asynchronously from Node.js using a dynamic JavaScript class interface, supports setting and getting Python-side properties, and enables receiving streamed partial results from long-running Python operations.
|
||||||
|
|
||||||
|
## How to Use
|
||||||
|
|
||||||
|
### JavaScript Usage Example
|
||||||
|
|
||||||
|
```js
|
||||||
|
import ChatModel from "./python_bindings.js";
|
||||||
|
|
||||||
|
async function runExample() {
|
||||||
|
|
||||||
|
const chat = new ChatModel({ model: "something" });
|
||||||
|
|
||||||
|
// Set properties dynamically
|
||||||
|
await chat.setProperty("model", "something");
|
||||||
|
await chat.setProperty("tokenizer", 123);
|
||||||
|
|
||||||
|
// Get properties from Python backend
|
||||||
|
const modelResponse = await chat.getModelPath();
|
||||||
|
console.log(modelResponse); // { Model: "something" }
|
||||||
|
|
||||||
|
const tokenizerResponse = await chat.getTokenizer();
|
||||||
|
console.log(tokenizerResponse); // { Tokenizer: 123 }
|
||||||
|
|
||||||
|
// Call Python methods asynchronously
|
||||||
|
let response = await chat.increment({ by: 5 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
response = await chat.increment({ by: 2 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
response = await chat.increment({ by: 2 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
// Listen for streamed partial results
|
||||||
|
chat.onMessage(function(data) {
|
||||||
|
console.log("Streamed data:", data);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Call method that streams partial results
|
||||||
|
await chat.testStream();
|
||||||
|
|
||||||
|
// Cleanly terminate Python subprocess
|
||||||
|
chat.end();
|
||||||
|
}
|
||||||
|
|
||||||
|
runExample();
|
||||||
|
```
|
||||||
|
|
||||||
|
## How to Write the Python Controller
|
||||||
|
|
||||||
|
The Python controller defines the backend logic and exposes methods callable from Node.js. It should extend the provided `BaseController` class and implement any methods you want to call from Node.js.
|
||||||
|
|
||||||
|
### Controller Structure Example
|
||||||
|
|
||||||
|
```python
|
||||||
|
# python/controller.py
|
||||||
|
|
||||||
|
import time
|
||||||
|
from baseController import BaseController
|
||||||
|
|
||||||
|
class Controller(BaseController):
|
||||||
|
|
||||||
|
model = None
|
||||||
|
tokenizer = None
|
||||||
|
counter = 0
|
||||||
|
|
||||||
|
def getModelPath(self, params):
|
||||||
|
return {"Model": self.model}
|
||||||
|
|
||||||
|
def getTokenizer(self, params):
|
||||||
|
return {"Tokenizer": self.tokenizer}
|
||||||
|
|
||||||
|
def increment(self, params):
|
||||||
|
self.counter += params.get("by", 1)
|
||||||
|
return {"counter": self.counter}
|
||||||
|
|
||||||
|
def reset(self, params):
|
||||||
|
self.counter = 0
|
||||||
|
return {"counter": self.counter}
|
||||||
|
|
||||||
|
def testStream(self, params):
|
||||||
|
for i in range(5):
|
||||||
|
time.sleep(0.5)
|
||||||
|
self.send({"partial": f"step {i+1} complete"})
|
||||||
|
return {"counter": self.counter}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Important Details
|
||||||
|
|
||||||
|
- **Inheritance:** Your controller must inherit from `BaseController`.
|
||||||
|
- **Methods:** Each method takes a single `params` dictionary argument containing parameters passed from Node.js.
|
||||||
|
- **Return Value:** Methods return a JSON-serializable dictionary as a response.
|
||||||
|
- **Streaming:** Use `self.send(data)` within methods to send partial streaming data back to Node.js. The JavaScript side will receive these via the registered stream callback.
|
||||||
|
- **Properties:** Define class properties to maintain state accessible from both Python and Node.js via dynamic `setProperty` and `getProperty` calls.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
- Extend the Python `Controller` class to implement your backend logic.
|
||||||
|
- Methods receive parameters and return JSON-serializable results.
|
||||||
|
- Use `self.send()` to stream intermediate results when needed.
|
||||||
|
- From Node.js, call these methods via the binding class, passing parameters as objects and receiving results asynchronously.
|
||||||
|
- The binding handles JSON serialization, communication, and a persistent Python process lifecycle.
|
||||||
|
|
||||||
|
This design allows flexible and efficient integration between Node.js and Python for complex applications.
|
||||||
|
|||||||
116
create.js
Normal file
116
create.js
Normal file
@@ -0,0 +1,116 @@
|
|||||||
|
const readme = `# Python Binding for Node.js
|
||||||
|
|
||||||
|
## Description
|
||||||
|
|
||||||
|
This project enables seamless integration of a Python backend with a Node.js application by maintaining a persistent Python subprocess. It allows calling Python methods asynchronously from Node.js using a dynamic JavaScript class interface, supports setting and getting Python-side properties, and enables receiving streamed partial results from long-running Python operations.
|
||||||
|
|
||||||
|
## How to Use
|
||||||
|
|
||||||
|
### JavaScript Usage Example
|
||||||
|
|
||||||
|
\`\`\`js
|
||||||
|
import ChatModel from "./python_bindings.js";
|
||||||
|
|
||||||
|
async function runExample() {
|
||||||
|
|
||||||
|
const chat = new ChatModel({ model: "something" });
|
||||||
|
|
||||||
|
// Set properties dynamically
|
||||||
|
await chat.setProperty("model", "something");
|
||||||
|
await chat.setProperty("tokenizer", 123);
|
||||||
|
|
||||||
|
// Get properties from Python backend
|
||||||
|
const modelResponse = await chat.getModelPath();
|
||||||
|
console.log(modelResponse); // { Model: "something" }
|
||||||
|
|
||||||
|
const tokenizerResponse = await chat.getTokenizer();
|
||||||
|
console.log(tokenizerResponse); // { Tokenizer: 123 }
|
||||||
|
|
||||||
|
// Call Python methods asynchronously
|
||||||
|
let response = await chat.increment({ by: 5 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
response = await chat.increment({ by: 2 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
response = await chat.increment({ by: 2 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
// Listen for streamed partial results
|
||||||
|
chat.onMessage(function(data) {
|
||||||
|
console.log("Streamed data:", data);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Call method that streams partial results
|
||||||
|
await chat.testStream();
|
||||||
|
|
||||||
|
// Cleanly terminate Python subprocess
|
||||||
|
chat.end();
|
||||||
|
}
|
||||||
|
|
||||||
|
runExample();
|
||||||
|
\`\`\`
|
||||||
|
|
||||||
|
## How to Write the Python Controller
|
||||||
|
|
||||||
|
The Python controller defines the backend logic and exposes methods callable from Node.js. It should extend the provided \`BaseController\` class and implement any methods you want to call from Node.js.
|
||||||
|
|
||||||
|
### Controller Structure Example
|
||||||
|
|
||||||
|
\`\`\`python
|
||||||
|
# python/controller.py
|
||||||
|
|
||||||
|
import time
|
||||||
|
from baseController import BaseController
|
||||||
|
|
||||||
|
class Controller(BaseController):
|
||||||
|
|
||||||
|
model = None
|
||||||
|
tokenizer = None
|
||||||
|
counter = 0
|
||||||
|
|
||||||
|
def getModelPath(self, params):
|
||||||
|
return {"Model": self.model}
|
||||||
|
|
||||||
|
def getTokenizer(self, params):
|
||||||
|
return {"Tokenizer": self.tokenizer}
|
||||||
|
|
||||||
|
def increment(self, params):
|
||||||
|
self.counter += params.get("by", 1)
|
||||||
|
return {"counter": self.counter}
|
||||||
|
|
||||||
|
def reset(self, params):
|
||||||
|
self.counter = 0
|
||||||
|
return {"counter": self.counter}
|
||||||
|
|
||||||
|
def testStream(self, params):
|
||||||
|
for i in range(5):
|
||||||
|
time.sleep(0.5)
|
||||||
|
self.send({"partial": f"step {i+1} complete"})
|
||||||
|
return {"counter": self.counter}
|
||||||
|
\`\`\`
|
||||||
|
|
||||||
|
### Important Details
|
||||||
|
|
||||||
|
- **Inheritance:** Your controller must inherit from \`BaseController\`.
|
||||||
|
- **Methods:** Each method takes a single \`params\` dictionary argument containing parameters passed from Node.js.
|
||||||
|
- **Return Value:** Methods return a JSON-serializable dictionary as a response.
|
||||||
|
- **Streaming:** Use \`self.send(data)\` within methods to send partial streaming data back to Node.js. The JavaScript side will receive these via the registered stream callback.
|
||||||
|
- **Properties:** Define class properties to maintain state accessible from both Python and Node.js via dynamic \`setProperty\` and \`getProperty\` calls.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
- Extend the Python \`Controller\` class to implement your backend logic.
|
||||||
|
- Methods receive parameters and return JSON-serializable results.
|
||||||
|
- Use \`self.send()\` to stream intermediate results when needed.
|
||||||
|
- From Node.js, call these methods via the binding class, passing parameters as objects and receiving results asynchronously.
|
||||||
|
- The binding handles JSON serialization, communication, and a persistent Python process lifecycle.
|
||||||
|
|
||||||
|
This design allows flexible and efficient integration between Node.js and Python for complex applications.
|
||||||
|
`;
|
||||||
|
|
||||||
|
console.log(readme);
|
||||||
|
|
||||||
|
import { writeFileSync } from "fs";
|
||||||
|
|
||||||
|
writeFileSync("README.md", readme);
|
||||||
33
index.js
Normal file
33
index.js
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
import ChatModel from "./python_bindings.js";
|
||||||
|
|
||||||
|
const chat = new ChatModel({ "model":"something" });
|
||||||
|
|
||||||
|
//await chat.setProperty( "model", "something");
|
||||||
|
await chat.setProperty( "tokenizer", 123 );
|
||||||
|
|
||||||
|
|
||||||
|
const modelResponse = await chat.getModelPath();
|
||||||
|
console.log(modelResponse); // { model: "something" }
|
||||||
|
|
||||||
|
const tokenizerResponse = await chat.getTokenizer();
|
||||||
|
console.log(tokenizerResponse); // { model: 123 }
|
||||||
|
|
||||||
|
let response = await chat.increment({ by: 5 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
response = await chat.increment({ by: 2 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
response = await chat.increment({ by: 2 });
|
||||||
|
console.log("Incremented counter:", response.counter);
|
||||||
|
|
||||||
|
chat.onMessage( function( data ) {
|
||||||
|
|
||||||
|
console.log( data );
|
||||||
|
|
||||||
|
} );
|
||||||
|
|
||||||
|
chat.testStream();
|
||||||
|
|
||||||
|
chat.end();
|
||||||
|
|
||||||
9
package.json
Normal file
9
package.json
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
{
|
||||||
|
"type": "module",
|
||||||
|
"scripts": {
|
||||||
|
"postinstall": "node scripts/install-python-deps.js"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"python-shell": "^5.0.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
1
python-path.txt
Normal file
1
python-path.txt
Normal file
@@ -0,0 +1 @@
|
|||||||
|
/usr/bin/python3.9
|
||||||
35
python/baseController.py
Normal file
35
python/baseController.py
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
class BaseController:
|
||||||
|
|
||||||
|
def set_stream_func(self, stream_func):
|
||||||
|
self._stream_func = stream_func
|
||||||
|
|
||||||
|
def send(self, data):
|
||||||
|
if getattr(self, "_stream_func", None) is not None:
|
||||||
|
self._stream_func(data)
|
||||||
|
|
||||||
|
def setProperty(self, params):
|
||||||
|
name = params.get("name")
|
||||||
|
value = params.get("value")
|
||||||
|
|
||||||
|
if not isinstance(name, str):
|
||||||
|
return {"error": "Property name must be a string"}
|
||||||
|
|
||||||
|
if not hasattr(self, name):
|
||||||
|
return {"error": f"Property '{name}' does not exist"}
|
||||||
|
|
||||||
|
setattr(self, name, value)
|
||||||
|
|
||||||
|
return {"success": True}
|
||||||
|
|
||||||
|
|
||||||
|
def getProperty(self, params):
|
||||||
|
|
||||||
|
name = params.get("name")
|
||||||
|
|
||||||
|
if not hasattr(self, name):
|
||||||
|
|
||||||
|
return {"error": f"Property '{name}' does not exist"}
|
||||||
|
|
||||||
|
value = getattr(self, name)
|
||||||
|
|
||||||
|
return {"value": value}
|
||||||
41
python/controller.py
Normal file
41
python/controller.py
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
import time
|
||||||
|
|
||||||
|
from baseController import BaseController
|
||||||
|
|
||||||
|
class Controller(BaseController):
|
||||||
|
|
||||||
|
model = None
|
||||||
|
|
||||||
|
tokenizer = None
|
||||||
|
|
||||||
|
counter = 0
|
||||||
|
|
||||||
|
def getModelPath(self, params):
|
||||||
|
|
||||||
|
return {"Model": self.model}
|
||||||
|
|
||||||
|
def getTokenizer(self, params):
|
||||||
|
|
||||||
|
return {"Tokenizer": self.tokenizer}
|
||||||
|
|
||||||
|
def increment(self, params):
|
||||||
|
|
||||||
|
self.counter += params.get("by", 1)
|
||||||
|
|
||||||
|
return {"counter": self.counter}
|
||||||
|
|
||||||
|
def reset(self, params):
|
||||||
|
|
||||||
|
self.counter = 0
|
||||||
|
|
||||||
|
return {"counter": self.counter}
|
||||||
|
|
||||||
|
def testStream(self, params):
|
||||||
|
|
||||||
|
for i in range(5):
|
||||||
|
|
||||||
|
time.sleep(0.5)
|
||||||
|
|
||||||
|
self.send({"partial": f"step {i+1} complete"})
|
||||||
|
|
||||||
|
return {"counter": self.counter}
|
||||||
5
python/index.py
Normal file
5
python/index.py
Normal file
@@ -0,0 +1,5 @@
|
|||||||
|
from router import Router
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
router = Router()
|
||||||
|
router.run()
|
||||||
42
python/router.py
Normal file
42
python/router.py
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
import time
|
||||||
|
from controller import Controller
|
||||||
|
import sys
|
||||||
|
import json
|
||||||
|
|
||||||
|
class Router:
|
||||||
|
def __init__(self):
|
||||||
|
self.controller = Controller()
|
||||||
|
self.controller.send = self.send_stream # inject send method here
|
||||||
|
|
||||||
|
def send_response(self, response):
|
||||||
|
print(json.dumps(response))
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
def send_stream(self, stream_data):
|
||||||
|
print("__STREAM__" + json.dumps(stream_data))
|
||||||
|
sys.stdout.flush()
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
for line in sys.stdin:
|
||||||
|
if not line.strip():
|
||||||
|
continue
|
||||||
|
|
||||||
|
try:
|
||||||
|
message = json.loads(line)
|
||||||
|
method = message.get("method")
|
||||||
|
params = message.get("params", {})
|
||||||
|
|
||||||
|
if not hasattr(self.controller, method):
|
||||||
|
self.send_response({"error": f"Method '{method}' not found in ./python/controller.py"})
|
||||||
|
continue
|
||||||
|
|
||||||
|
method_to_call = getattr(self.controller, method)
|
||||||
|
|
||||||
|
# Call method WITHOUT stream_func argument,
|
||||||
|
# controller methods use self.send() internally
|
||||||
|
result = method_to_call(params)
|
||||||
|
|
||||||
|
self.send_response({"result": result})
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
self.send_response({"error": str(e)})
|
||||||
4
python/test_import.py
Normal file
4
python/test_import.py
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
from controller import Controller
|
||||||
|
|
||||||
|
c = Controller()
|
||||||
|
print(c.increment({"by": 3}))
|
||||||
184
python_bindings.js
Normal file
184
python_bindings.js
Normal file
@@ -0,0 +1,184 @@
|
|||||||
|
import { PythonShell } from "python-shell";
|
||||||
|
|
||||||
|
export default class Bindings {
|
||||||
|
constructor(initialProps) {
|
||||||
|
|
||||||
|
this.pythonShell = new PythonShell("./python/index.py", { mode: "text" });
|
||||||
|
this.responseQueue = new Array();
|
||||||
|
this.sendQueue = new Array();
|
||||||
|
this.isSending = false;
|
||||||
|
this.streamCallback = null;
|
||||||
|
|
||||||
|
this.pythonShell.on("message", function(message) {
|
||||||
|
if (typeof message === "string" && message.startsWith("__STREAM__")) {
|
||||||
|
var dataStr = message.substring(10);
|
||||||
|
var data = null;
|
||||||
|
|
||||||
|
try {
|
||||||
|
data = JSON.parse(dataStr);
|
||||||
|
} catch (err) {
|
||||||
|
console.error("Invalid streamed JSON:", dataStr);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (this.streamCallback) {
|
||||||
|
this.streamCallback(data);
|
||||||
|
}
|
||||||
|
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var response = null;
|
||||||
|
|
||||||
|
try {
|
||||||
|
response = JSON.parse(message);
|
||||||
|
} catch (err) {
|
||||||
|
console.error("Invalid JSON response:", message);
|
||||||
|
var resolve = this.responseQueue.shift();
|
||||||
|
|
||||||
|
if (resolve) {
|
||||||
|
resolve({ error: "Invalid JSON from Python" });
|
||||||
|
}
|
||||||
|
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var resolve = this.responseQueue.shift();
|
||||||
|
|
||||||
|
if (resolve) {
|
||||||
|
resolve(response);
|
||||||
|
}
|
||||||
|
|
||||||
|
this.processSendQueue();
|
||||||
|
|
||||||
|
}.bind(this));
|
||||||
|
|
||||||
|
this.pythonShell.on("error", function(err) {
|
||||||
|
console.error("PythonShell error:", err);
|
||||||
|
});
|
||||||
|
|
||||||
|
this.pythonShell.on("stderr", function(stderr) {
|
||||||
|
console.error("PythonShell stderr:", stderr);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Set initial properties if any
|
||||||
|
if (initialProps && typeof initialProps === "object") {
|
||||||
|
var keys = Object.keys(initialProps);
|
||||||
|
|
||||||
|
for (var i = 0; i < keys.length; i++) {
|
||||||
|
var key = keys[i];
|
||||||
|
var value = initialProps[key];
|
||||||
|
|
||||||
|
this.callMethod("setProperty", key, value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return new Proxy(this, {
|
||||||
|
get: function(target, prop) {
|
||||||
|
if (typeof prop === "string") {
|
||||||
|
if (prop === "end") {
|
||||||
|
return target.end.bind(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (prop === "onMessage") {
|
||||||
|
return target.onMessage.bind(target);
|
||||||
|
}
|
||||||
|
|
||||||
|
return function() {
|
||||||
|
var args = new Array(arguments.length);
|
||||||
|
|
||||||
|
for (var i = 0; i < arguments.length; i++) {
|
||||||
|
args[i] = arguments[i];
|
||||||
|
}
|
||||||
|
|
||||||
|
return target.callMethod(prop, ...args);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return undefined;
|
||||||
|
},
|
||||||
|
|
||||||
|
set: function(target, prop, value) {
|
||||||
|
return target.callMethod("setProperty", prop, value)
|
||||||
|
.then(function() {
|
||||||
|
return true;
|
||||||
|
})
|
||||||
|
.catch(function(err) {
|
||||||
|
console.error(err);
|
||||||
|
return false;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
onMessage(callback) {
|
||||||
|
this.streamCallback = callback;
|
||||||
|
}
|
||||||
|
|
||||||
|
callMethod(method) {
|
||||||
|
var args = new Array(arguments.length - 1);
|
||||||
|
|
||||||
|
for (var i = 1; i < arguments.length; i++) {
|
||||||
|
args[i - 1] = arguments[i];
|
||||||
|
}
|
||||||
|
|
||||||
|
return new Promise(function(resolve, reject) {
|
||||||
|
this.responseQueue.push(function(response) {
|
||||||
|
if (response.error) {
|
||||||
|
reject(new Error(response.error));
|
||||||
|
} else {
|
||||||
|
resolve(response.result !== undefined ? response.result : response);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
var params = {};
|
||||||
|
|
||||||
|
if (method === "setProperty" && args.length === 2) {
|
||||||
|
params = { name: args[0], value: args[1] };
|
||||||
|
} else if (args.length === 1 && typeof args[0] === "object") {
|
||||||
|
params = args[0];
|
||||||
|
} else if (args.length > 0) {
|
||||||
|
params = { args: args };
|
||||||
|
}
|
||||||
|
|
||||||
|
this.sendQueue.push({ method: method, params: params });
|
||||||
|
|
||||||
|
this.processSendQueue();
|
||||||
|
|
||||||
|
}.bind(this));
|
||||||
|
}
|
||||||
|
|
||||||
|
processSendQueue() {
|
||||||
|
if (this.isSending === true) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (this.sendQueue.length === 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
var message = this.sendQueue.shift();
|
||||||
|
|
||||||
|
this.isSending = true;
|
||||||
|
|
||||||
|
try {
|
||||||
|
this.pythonShell.send(JSON.stringify(message));
|
||||||
|
} catch (err) {
|
||||||
|
var resolve = this.responseQueue.shift();
|
||||||
|
|
||||||
|
if (resolve) {
|
||||||
|
resolve({ error: err.message });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
this.isSending = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
end() {
|
||||||
|
this.pythonShell.end(function(err) {
|
||||||
|
if (err !== undefined && err !== null) {
|
||||||
|
console.error("Error ending PythonShell:", err);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
3
requirements.txt
Normal file
3
requirements.txt
Normal file
@@ -0,0 +1,3 @@
|
|||||||
|
torch
|
||||||
|
transformers
|
||||||
|
peft
|
||||||
46
router.js
Normal file
46
router.js
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
import { spawn } from "child_process";
|
||||||
|
|
||||||
|
export class Router {
|
||||||
|
|
||||||
|
constructor () {
|
||||||
|
this.callbacks = { };
|
||||||
|
this.stdoutBuffer = "";
|
||||||
|
this.onmessage = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async callMethod ( method, params = { } ) {
|
||||||
|
const child = spawn( "python3", [ "index.py", method, JSON.stringify( params ) ] );
|
||||||
|
|
||||||
|
return await new Promise( ( resolve, reject ) => {
|
||||||
|
let result = "";
|
||||||
|
|
||||||
|
child.stdout.on( "data", ( data ) => {
|
||||||
|
const lines = data.toString().split( "\n" );
|
||||||
|
|
||||||
|
for ( const line of lines ) {
|
||||||
|
if ( line.startsWith( "__STREAM__" ) ) {
|
||||||
|
const payload = line.substring( 10 ).trim();
|
||||||
|
|
||||||
|
if ( this.onmessage ) {
|
||||||
|
this.onmessage( JSON.parse( payload ) );
|
||||||
|
}
|
||||||
|
} else if ( line.trim() ) {
|
||||||
|
result += line.trim();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} );
|
||||||
|
|
||||||
|
child.stderr.on( "data", ( err ) => {
|
||||||
|
reject( new Error( err.toString() ) );
|
||||||
|
} );
|
||||||
|
|
||||||
|
child.on( "close", () => {
|
||||||
|
try {
|
||||||
|
resolve( JSON.parse( result ) );
|
||||||
|
} catch ( err ) {
|
||||||
|
reject( new Error( "Failed to parse response: " + result ) );
|
||||||
|
}
|
||||||
|
} );
|
||||||
|
} );
|
||||||
|
}
|
||||||
|
}
|
||||||
50
scripts/install-python-deps.js
Normal file
50
scripts/install-python-deps.js
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
import { execSync } from "child_process";
|
||||||
|
import { writeFileSync } from "fs";
|
||||||
|
import path from "path";
|
||||||
|
|
||||||
|
const candidates = [
|
||||||
|
"/usr/bin/python3.9",
|
||||||
|
"python3",
|
||||||
|
"python"
|
||||||
|
];
|
||||||
|
|
||||||
|
const pythonPathFile = path.resolve(process.cwd(), "python-path.txt");
|
||||||
|
|
||||||
|
function checkPython(pythonCmd) {
|
||||||
|
try {
|
||||||
|
execSync(`${pythonCmd} --version`, { stdio: "ignore" });
|
||||||
|
execSync(`${pythonCmd} -m pip --version`, { stdio: "ignore" });
|
||||||
|
return true;
|
||||||
|
} catch {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function installDeps(pythonCmd) {
|
||||||
|
try {
|
||||||
|
console.log(`Using Python command: ${pythonCmd}`);
|
||||||
|
execSync(`${pythonCmd} -m pip install -r requirements.txt`, { stdio: "inherit" });
|
||||||
|
writeFileSync(pythonPathFile, pythonCmd, "utf-8");
|
||||||
|
console.log(`✅ Python packages installed successfully. Saved Python path to ${pythonPathFile}`);
|
||||||
|
process.exit(0);
|
||||||
|
} catch (err) {
|
||||||
|
console.error("Failed to install Python packages with", pythonCmd);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function main() {
|
||||||
|
for (const cmd of candidates) {
|
||||||
|
if (checkPython(cmd)) {
|
||||||
|
installDeps(cmd);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
console.error(
|
||||||
|
"\n⚠️ Python or pip not found or pip is broken.\n" +
|
||||||
|
"Please install Python 3 and pip manually, then run:\n\n" +
|
||||||
|
" pip install -r requirements.txt\n"
|
||||||
|
);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
|
||||||
|
main();
|
||||||
Reference in New Issue
Block a user