lite/global.h

using lite::DecryptionFunc = std::function<std::vector<uint8_t>(const void*, size_t, const std::vector<uint8_t>&)>

Model decryption function.

Param const

[in] void* is the decrypted model memory pointer

Param size_t

[in] the size the decrypted model memory in byte

Param const

[in] std::vector<uint8_t>& the decryption key vector

bool lite::register_decryption_and_key(std::string decrypt_name, const DecryptionFunc &func, const std::vector<uint8_t> &key)

register a custom decryption method and key to lite.

参数
  • decrypt_name[in] the name of the decryption, which will act as the hash key to find the decryption method.

  • func[in] the decryption function, which will decrypt the model with the registered key, return a vector that contain the decrypted model.

  • key[in] the decryption key of the method

bool lite::update_decryption_or_key(std::string decrypt_name, const DecryptionFunc &func, const std::vector<uint8_t> &key)

update decryption function or key of a custom decryption method.

参数
  • decrypt_name[in] the name of the decryption, which will act as the hash key to find the decryption method.

  • func[in] the decryption function, which will decrypt the model with the registered key, return a vector that contain the decrypted model. if function is nullptr, it will not be updated.

  • key[in] the decryption key of the method, if the size of key is zero, it will not be updated

using lite::ParseInfoFunc = std::function<bool(const void*, size_t, const std::string model_name, Config &config, NetworkIO &network_io, std::unordered_map<std::string, LiteAny> &isolated_config_map, std::string &extra_info)>

Model information parse function.

Param const

[in] void* is the information memory

Param size_t

[in] the size the information memory

Param const

[in] std::string the model name used for check whether the infomation match the model

Param Config

the model config, ParseInfoFunc can fill it with the information in json, the config will influence Network loading later

Param NetworkIO

the model IO, ParseInfoFunc can fill it with the information in json, the networkio will influence Network forwarding later

Param std::unordered_map<std::string, LiteAny>&

[in] isolated_config_map, the other config not inclue in config and networkIO, ParseInfoFunc can fill it with the information in json, now support: “device_id” : int, default 0 “number_threads” : uint32_t, default 1 “is_inplace_model” : bool, default false “use_tensorrt” : bool, default false

bool lite::register_parse_info_func(std::string info_type, const ParseInfoFunc &parse_func)

register a custom parser function to lite.

参数
  • info_type[in] the name of the parser function, which will act as the hash key to find the parser method.

  • parse_func[in] the parser function, which will parse the given information and modify the Network Config and IO.

void lite::try_coalesce_all_free_memory()

try to coalesce all free memory in megenine

void lite::set_loader_lib_path(const std::string &loader_path)

Set the loader to the lite.

参数

loader_path – is the file path which store the cache

void lite::set_persistent_cache(const std::string &cache_path, bool always_sync = false)

Set the algo policy cache file for CPU/CUDA …

参数
  • cache_path – is the file path which store the cache

  • always_sync – sync the cache when model run

void lite::dump_persistent_cache(const std::string &cache_path)

dump the PersistentCache policy cache to file, if the network is set to profile when forward, though this the algo policy will dump to file

void lite::get_version(int &major, int &minor, int &patch)

Get version.

void lite::set_log_level(LiteLogLevel level)

Set the current log level.

参数

level[in] The new log level

LiteLogLevel lite::get_log_level()

Get the current log level.

返回

The current log level

size_t lite::get_device_count(LiteDeviceType device_type)

Get device count.

参数

device_type[in] device type

返回

the device count

bool lite::register_memory_pair(void *vir_ptr, void *phy_ptr, size_t length, LiteDeviceType device, LiteBackend backend = LiteBackend::LITE_DEFAULT)

register the physical and virtual address pair to the mge, some device need the map from physical to virtual.

bool lite::clear_memory_pair(void *vir_ptr, void *phy_ptr, LiteDeviceType device, LiteBackend backend = LiteBackend::LITE_DEFAULT)

clear the physical and virtual address pair in mge.

void *lite::lookup_physic_ptr(void *vir_ptr, LiteDeviceType device, LiteBackend backend)

get the physic address by the virtual address in mge.