《AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks》特征交叉论文阅读

背景

这是一篇利用多头attention机制来做特征交叉的论文

模型结构

AutoInt的模型结构如上图所示，搞模型包含 Embedding Layer、Interacting Layer、Output Layer三个部分，其中Embedding Layer和Output Layer和普通模型没什么区别，主要介绍一下Interacting Layer

Interacting Layer是改名实现特征交叉的部分，其实现特征交叉的过程可以分为以下几步：

1. 计算特征之间的相关性

2. 根据相关系数，计算特征交叉

3. 拼接多个头的特征交叉结果

4. 为了保留原始特征，用一个残差结构拼接原始特征和交叉后的特征

最终在输出层，拼接所有交叉后的特征过sigmoid函数：

本栏目推荐文章
• 《RAPL: A Relation-Aware Prototype Learning Approach for Few-Shot Document-Level Relation Extraction》阅读笔记
• 【五期李伟平】CCF-A（AAAI'21）Game of Gradients: Mitigating Irrelevant Clients in Federated Learning
• Learning Dynamic Query Combinations for Transformer-based Object** Detection and Segmentation论文阅读笔记
• 1.9 Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation 基于语义分割遥感图像的模型
• 【略读论文|大模型相关】Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models
• 《An End-to-end Model for Entity-level Relation Extraction using Multi-instance Learning》阅读笔记
• CF940F Machine Learning题解
• CF940FMachine Learning题解
• AMOS: Enabling Automatic Mapping for Tensor Computations On Spatial Accelerators with Hardware Abstraction
• 《Ensemble deep learning: A review》阅读笔记

Self-Attentive Interaction Attentive Automatic Learningself-attentive interaction attentive automatic self-attentive attentive attentive attention networks graph automatic recommendation session-based attentive session interaction evaluation automatic generated language interaction detecting encrypted malicious optimization abstraction tensorized automatic