基于BERT的语义感知二进制代码表示

需积分: 5 4 下载量 24 浏览量 更新于2024-07-07 收藏 3.65MB DOCX 举报
2021_Semantic-aware Binary Code Representation with BERT 本文提出了一种基于BERT的语义感知二进制代码表示方法,称为DeepSemantic。该方法通过使用BERT架构,能够自动地生成二进制代码的语义感知表示,从而提高二进制分析的准确性和效率。 DeepSemantic方法的提出背景是,由于二进制代码分析的重要性日益增加,例如bug发现、恶意软件分析和代码克隆检测等。但是,现有的基于机器学习的二进制分析方法仍然存在一些缺陷,例如需要手动制作分析算法的具体内容,或者需要重新创造模型以适应不同类型的二进制分析。 为解决这些问题,DeepSemantic方法引入了平衡良好的指令归一化,以保留丰富的信息,同时将词汇外(OOV)问题降到最低。此外,DeepSemantic方法还利用了BERT架构的精髓,重新利用了一个预先训练好的通用模型,该模型可随时作为一次性处理,然后通过一个微调过程快速应用特定的下游任务。 在实验中,DeepSemantic方法在两个下游任务中都取得了很好的结果,即二进制相似性比较和编译器出处预测。实验结果表明,二进制相似性模型优于两个最先进的二进制相似性工具,DeepBinDiff和SAFE,平均分别为49.84%和15.83%。 在现代计算环境中,二进制分析在实现广泛的流行用例中起着举足轻重的作用,例如代码克隆或软件剽窃检测、分布式软件的漏洞发现、恶意软件检测和分类、程序修复或补丁分析、以及用于数字取证的工具链来源等。因此,DeepSemantic方法的提出将对二进制分析领域产生深远的影响。 在技术层面,DeepSemantic方法的提出也将推动二进制分析技术的发展。例如,基于BERT的语义感知二进制代码表示将能够更好地捕捉二进制代码的语义信息,从而提高二进制分析的准确性和效率。此外,DeepSemantic方法也将推动机器学习技术在二进制分析领域的应用,例如使用预先训练好的通用模型来快速应用特定的下游任务。 DeepSemantic方法的提出将对二进制分析领域产生深远的影响,推动二进制分析技术的发展,并提高二进制分析的准确性和效率。

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. spyder 5.4.1 requires pyqt5<5.16, which is not installed. spyder 5.4.1 requires pyqtwebengine<5.16, which is not installed. Successfully installed aiofiles-23.1.0 altair-4.2.2 blinker-1.6.2 cachetools-5.3.1 chardet-5.1.0 cmake-3.26.3 cpm_kernels-1.0.11 fastapi-0.95.2 ffmpy-0.3.0 gitdb-4.0.10 gitpython-3.1.31 gradio-3.32.0 gradio-client-0.2.5 h11-0.14.0 httpcore-0.17.2 httpx-0.24.1 latex2mathml-3.76.0 linkify-it-py-2.0.2 lit-16.0.5 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 mdtex2html-1.2.0 mdurl-0.1.2 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 orjson-3.8.14 protobuf-3.20.3 pydantic-1.10.8 pydeck-0.8.1b0 pydub-0.25.1 pygments-2.15.1 pympler-1.0.1 python-multipart-0.0.6 rich-13.4.1 semantic-version-2.10.0 sentencepiece-0.1.99 smmap-5.0.0 starlette-0.27.0 streamlit-1.22.0 streamlit-chat-0.0.2.2 torch-2.0.1 transformers-4.27.1 triton-2.0.0 tzlocal-5.0.1 uc-micro-py-1.0.2 uvicorn-0.22.0 validators-0.20.0 websockets-11.0.3 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv 解释下

2023-06-02 上传

CREATE TEMPORARY FUNCTION UUIDUDF AS 'com.haierubic.bigdata.commons.udf.UUIDUDF' . . . . . . . . . . . . . . . . . > USING JAR 'oss://datalake-01.cn-beijing.oss-dls.aliyuncs.com/config/bigdata-hiveudf-2.1-jar-with-dependencies.jar'; Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [hdop_upbrain] does not have [TEMPUDFADMIN] privilege on [global=uuidudf] (state=42000,code=40000) 0: jdbc:hive2://10.204.11.45:10000> CREATE FUNCTION UUIDUDF AS 'com.haierubic.bigdata.commons.udf.UUIDUDF' . . . . . . . . . . . . . . . . . > USING JAR 'oss://datalake-01.cn-beijing.oss-dls.aliyuncs.com/config/bigdata-hiveudf-2.1-jar-with-dependencies.jar'; INFO : Compiling command(queryId=hive_20230602122812_92858e15-5136-4e7d-9f51-3020f864aef2): CREATE FUNCTION UUIDUDF AS 'com.haierubic.bigdata.commons.udf.UUIDUDF' USING JAR 'oss://datalake-01.cn-beijing.oss-dls.aliyuncs.com/config/bigdata-hiveudf-2.1-jar-with-dependencies.jar' INFO : Concurrency mode is disabled, not creating a lock manager INFO : Semantic Analysis Completed (retrial = false) INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null) INFO : Completed compiling command(queryId=hive_20230602122812_92858e15-5136-4e7d-9f51-3020f864aef2); Time taken: 0.067 seconds INFO : Concurrency mode is disabled, not creating a lock manager INFO : Executing command(queryId=hive_20230602122812_92858e15-5136-4e7d-9f51-3020f864aef2): CREATE FUNCTION UUIDUDF AS 'com.haierubic.bigdata.commons.udf.UUIDUDF' USING JAR 'oss://datalake-01.cn-beijing.oss-dls.aliyuncs.com/config/bigdata-hiveudf-2.1-jar-with-dependencies.jar' INFO : Starting task [Stage-0:FUNC] in serial mode INFO : Added [/tmp/b11e4544-4a21-4dcf-87c5-fff8d91021e9_resources/bigdata-hiveudf-2.1-jar-with-dependencies.jar] to class path INFO : Added resources: [oss://datalake-01.cn-beijing.oss-dls.aliyuncs.com/config/bigdata-hiveudf-2.1-jar-with-dependencies.jar] INFO : Completed executing command(queryId=hive_20230602122812_92858e15-5136-4e7d-9f51-3020f864aef2); Time taken: 0.789 seconds INFO : OK INFO : Concurrency mode is disabled, not creating a lock manager No rows affected (0.881 seconds)

2023-06-03 上传