【5A文】迁移学习算法研究.ppt

5A文】迁移学习算法研究,Training Data,What if…,2019/1/12,2,传统监督机器学习(1/2),[from Prof. Qiang Yang],传统监督机器学习(2/2),2019/1/12,3,传统监督学习,迁移学习,2019/1/12,4,实际应用学习场景,,迁移 学习,运用已有的知识对不同但相关领域问题进行求解的一种新的机器学习方法 放宽了传统机器学习的两个基本假设,迁移学习场景(1/4),2019/1/12,5,迁移学习场景无处不在,迁移学习场景(2/4),异构特征空间,2019/1/12,6,,,The apple is the pomaceous fruit of the apple tree, species Malus domestica in the rose family Rosaceae .,Banana is the common name for a type of fruit and also the herbaceous plants of the genus Musa which produce this commonly eaten fruit .,Training: Text,Future: Images,Apples,Bananas,[from Prof. Qiang Yang],Xin Jin, Fuzhen Zhuang, Sinno Jialin Pan, Changying Du, Ping Luo, Qing He: Heterogeneous Multi-task Semantic Feature Learning for Classification. CIKM 2015 : 1847-1850.,Test,Test,Training,Training,,Classifier,,,Classifier,,72.65%,DVD,Electronics,Electronics,84.60%,,Electronics,Drop!,迁移学习场景(3/4),2019/1/12,7,[from Prof. Qiang Yang],迁移学习场景(4/4),2019/1/12,8,,DVD,Electronics,Book,Kitchen,Clothes,Video game,Fruit,Hotel,Tea,Impractical!,[from Prof. Qiang Yang],Outline,Concept Learning for Transfer Learning Concept Learning based on Non-negative Matrix Tri-factorization for Transfer Learning Concept Learning based on Probabilistic Latent Semantic Analysis for Transfer Learning Transfer Learning using Auto-encoders Transfer Learning from Multiple Sources with Autoencoder Regularization Supervised Representation Learning: Transfer Learning with Deep Auto-encoders,2019/1/12,9,Concept Learning based on Non-negative Matrix Tri-factorization for Transfer Learning,2019/1/12,Concept Learning for Transfer Learning,10,Introduction,2019/1/12,Concept Learning for Transfer Learning,11,Many traditional learning techniques work well only under the assumption: Training and test data follow the same distribution,Training (labeled),Classifier,Test (unlabeled),,,Enterprise News Classification: including the classes “Product Announcement”, “Business scandal”, “Acquisition”, … …,Product announcement: HP's just-released LaserJet Pro P1100 printer and the LaserJet Pro M1130 and M1210 multifunction printers, price … performance .,Announcement for Lenovo ThinkPad ThinkCentre – price $150 off Lenovo K300 desktop using coupon code . Lenovo ThinkPad ThinkCentre – price $200 off Lenovo IdeaPad U450p laptop using. .their performance,HP news,Lenovo news,Different distribution,Fail !,,Motivation (1/3),2019/1/12,Concept Learning for Transfer Learning,12,Example Analysis,Product announcement: HP's just-released LaserJet Pro P1100 printer and the LaserJet Pro M1130 and M1210 multifunction printers, price … performance .,Announcement for Lenovo ThinkPad ThinkCentre – price $150 off Lenovo K300 desktop using coupon code . Lenovo ThinkPad ThinkCentre – price $200 off Lenovo IdeaPad U450p laptop using. .their performance,HP news,Lenovo news,Product,word concept,LaserJet, printer, price, performance,ThinkPad, ThinkCentre, price, performance,,,,,,Related,Product announcement,,document class:,Share some common words: announcement, price, performance …,indicate,,Motivation (2/3),2019/1/12,Concept Learning for Transfer Learning,13,Example Analysis:,The words expressing the same word concept are domain-dependent,,,Product,,,Product announcement,,,word concept,indicates,,,The association between word concepts and document classes is domain-independent,,Motivation (3/3),2019/1/12,Concept Learning for Transfer Learning,14,Further observations: Different domains may use same key words to express the same concept (denoted as identical concept) Different domains may also use different key words to express the same concept (denoted as alike concept) Different domains may also have their own distinct concepts (denoted as distinct concept) The identical and alike concepts are used as the shared concepts for knowledge transfer We try to model these three kinds of concepts simultaneously for transfer learning text classification,Preliminary Knowledge,2019/1/12,Concept Learning for Transfer Learning,15,Basic formula of matrix tri-factorization: where the input X is the word-document co-occurrence matrix,F,G,S,Previous method - MTrick in SDM 2010 (1/2),2019/1/12,Concept Learning for Transfer Learning,16,Sketch map of MTrick,,,Source domain Xs,,,Fs,Gs,Ft,Gt,Target domain Xt,S,,Knowledge Transfer,Considering the alike concepts,MTrick (2/2),Optimization problem for MTrick,2019/1/12,Concept Learning for Transfer Learning,17,G0 is the supervision information,,,,,the association S is shared as bridge to transfer knowledge,,,Dual Transfer Learning (Long et al., SDM 2012), considering identical and alike concepts,Triplex Transfer Learning (TriTL) (1/5),2019/1/12,Concept Learning for Transfer Learning,18,Further divide the word concepts into three kinds:,F1, identical concepts; F2, alike concepts; F3, distinct concepts,Input: s source domain Xr(1≤r≤s) with label information, t target domain Xr (s+1≤r≤s+t) We propose Triplex Transfer Learning framework based on matrix tri-factorization (TriTL for short),F1, S1。