Skip to content

Build TensorRT Serving Job

This section introduces how to customly build a TensorRT serving job.




func NewTRTServingJobBuilder() *TRTServingJobBuilder


TRTServingJobBuilder has following functions to custom your TensorRT serving job.

function description matches cli option
Name(name string) *TRTServingJobBuilder specify the job name --name
Namespace(namespace string) *TRTServingJobBuilder specify the namespace --namespace/-n
Command(args []string) *TRTServingJobBuilder specify the command -
GPUCount(count int) *TRTServingJobBuilder specify the gpu count --gpus
GPUMemory(memory int) *TRTServingJobBuilder specify the gpu memory(gpushare) --gpumemory
Image(image string) *TRTServingJobBuilder specify the image --image
ImagePullPolicy(policy string) *TRTServingJobBuilder specify the image pull policy --image-pull-policy
CPU(cpu string) *TRTServingJobBuilder specify the cpu limitation --cpu
Memory(memory string) *TRTServingJobBuilder specify the memory limitation --memory
Envs(envs map[string]string) *TRTServingJobBuilder specify the envs of containers --env
Replicas(count int) *TRTServingJobBuilder specify the replicas --replicas
EnableIstio() *TRTServingJobBuilder enable istio --enable-istio
ExposeService() *TRTServingJobBuilder expose service --expose-service
Version(version string) *TRTServingJobBuilder specify the version --version
Tolerations(tolerations []string) *TRTServingJobBuilder specify the node taint tolerations --toleration
NodeSelectors(selectors map[string]string) *TRTServingJobBuilder specify the node selectors --selector
Annotations(annotations map[string]string) *TRTServingJobBuilder specify the annotation --annotation
Datas(volumes map[string]string) *TRTServingJobBuilder specify the pvc which stores dataset --data
DataDirs(volumes map[string]string) *TRTServingJobBuilder specify the host path which stores dataset --data-dir
HttpPort(port int) *TRTServingJobBuilder specify the http service port --http-port
GrpcPort(port int) *TRTServingJobBuilder specify the grpc service port --grpc-port
MetricsPort(port int) *TRTServingJobBuilder specify the metric port --metric-port
ModelStore(store string) *TRTServingJobBuilder specify the path of storing model --model-store
AllowMetrics() *TRTServingJobBuilder enable metrics --allow-metrics
Build() (*Job, error) build the job -