并行计算完整PPT课件.ppt
41页
并行计算、集群与网格技术在虚拟天文台中应用探讨天津大学IBM中心 于策 内容大纲n并行计算技术n集群及相关技术n网格计算2021/3/92 如何满足不断增长的计算力需求？n用速度更快的硬件，也就是减少每一条指令所需时间n优化算法（或者优化编译）n用多个处理机用多个处理机(器器)同时解决一个问题同时解决一个问题q并行计算并行计算2021/3/93 串行计算与并行计算2021/3/94 并行的层次n程序级并行 n子程序级并行n语句级并行n操作级并行n微操作级并行并行粒度粗细2021/3/95 共享内存与分布式内存P=处理器M=内存S=网络交换设备PMP…PMPMS…2021/3/96 局部共享内存…SPMP…PMP…NodeNode2021/3/97 实现并行计算的常见方法n1、多线程程序n2、基于OpenMP的并行程序n3、基于消息传递的并行程序qPVMqMPI2021/3/98 XPVM2021/3/99 XPVM2021/3/910 并行计算环境简介nPVP（Parallel Vector Processor）q使用专用向量处理器q银河I，CrayT90，NEC的地球模拟器nSMP（Symmetric Multiprocessor）q同一主板上多个处理器可以等同访问共享存储器、I/O设备和操作系统服务qIBM RS6000。
nClusterq由很多连接在一起的独立计算机组成，像一个单独的集成的计算机一样协同工作q集群的节点可以是SMP（或者PVP）qCluster1350，蓝色基因2021/3/911 在虚拟天文台中的应用情形n处理实时性要求很高的大量观测数据n观测数据的并行可视化处理n并行数据挖掘n并行数据服务提供n2021/3/912 集群技术n集群概念最早由IBM于20世纪60年代提出n集群一般由高速网络连接起来的高性能工作站或PC机组成集群在工作中像一个统一的整合资源，所有节点使用单一界面2021/3/913 集群 Not a Cluster Cluster 2021/3/914 集群计算系统体系结构2021/3/915 集群相关技术n网络层q网络互联结构、通信协议、信号技术等 n节点机及操作系统层q高性能PC或工作站、分层或基于微内核的操作系统等 n集群系统管理层q资源管理、资源调度、负载平衡、并行I/O、安全等 n应用层q并行程序开发环境、串行应用、并行应用等2021/3/916 Cluster1350nCluster1350是IBM公司目标定位于高性能计算市场的Linux集群，包括一套完整的解决方案，集成了众多IBM与非IBM的先进的软硬件技术，有其特有的技术优势与强大的服务支持。
Cluster1350逻辑结构2021/3/918 Cluster1350节点硬件、网络和操作系统(天津大学IBM中心）n管理节点 qIBM eServer xSeries 345 (1)n计算节点 qIBM eServer xSeries 335 (16)n网络q千兆以太网n操作系统qRed Hat Linux 7.32021/3/919 CSM（Cluster System Management）nCSM是IBM公司开发，专门用于集群系统管理的中间件，在Cluster1350解决方案集成qCSM的设计思想与体系结构来自PSSP (IBM Parallel System Support Programs for AIX)与其它一些开源的集群管理软件还有一些中间件及技术，虽然不直接为用户服务，但构成了CSM的不可或缺的基础，包括RMC、SRC、RSCT等2021/3/920 CSM2021/3/921 Cluster1350系统管理n整个集群由单一结点控制q所有结点的n开机、关机、状态查询n显示远程控制台n安装操作系统n升级(安装)各结点系统及应用软件nq一个完整的集群只需一套外置输入/输出设备(键盘、鼠标、显示器)2021/3/922 Linux 集群上可用的任务调度系统nMosix（Multicomputer Operating System for UnIX ）q可以自动完成节点间进程迁移q需要重新编译Linux内核q对用户来说是透明的nPBS（Portable Batch System）q由NASA的Ames研究中心开发 q为了达到集群系统整体上的负载平衡，对各个节点上的任务进行调度，但不进行节点间任务迁移q不需要重新编译内核2021/3/923 集群上的应用程序（计算）开发n适于开发基于消息传递的并行应用程序q可以使用PVM/MPIn步骤：q配置并行计算的编译与运行环境n主要是rsh/sshq设计、编写、编译程序n使用PVM/MPI提供的程序库及编译环境q部署应用程序nftp、rcp、scp、CSM、NFS、GPFS等方式q运行程序2021/3/924 配置高可用集群n最大程度地减少服务中断，而不是容错。
qHearbeatqLVS（Linux Virtual Sever）qIBM 的 Tivoli 和 WebSphere 系列软件q……2021/3/925 Heartbeat 集群2021/3/926 LVS集群2021/3/927 集群在虚拟天文台中的角色n高性能计算q构成虚拟天文台网格环境中强有力的计算资源n高可用服务q在网格环境中提供稳定、可靠的数据、计算、调度等服务2021/3/928 网格计算 “Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations”2021/3/929 On Demand2021/3/930 理想的网格2021/3/931 理想的网格2021/3/932 InternetStorage ElementComputingElementWorker NodeUser InterfaceReplica Catalog+Replica ManagerResource Broker+Information IndexCertification AuthorityUser InterfaceTESTBED SITE 1TESTBED SITE 2Job Status:Front EndCertificate ServerTESTBED SITE 3Graphics by Aldo StentellaGraphics by Aldo StentellaThe picture above describes a DataGrid testbed with three sites and a Certification The picture above describes a DataGrid testbed with three sites and a Certification Authority. For space reasons we do not show all the machines of each site.Authority. For space reasons we do not show all the machines of each site.The user requests a certificate to the Certificate Authority web front-end.The user requests a certificate to the Certificate Authority web front-end.The certificate is produced and signed by the Certificate Server.The certificate is produced and signed by the Certificate Server.The user gets the certificate.The user gets the certificate.Now that the user has his/her certificate installed on his/her User Interface, he/she can Now that the user has his/her certificate installed on his/her User Interface, he/she can submit his/her job requests.submit his/her job requests. InternetStorage ElementComputingElementWorker NodeUser InterfaceReplica Catalog+Replica ManagerResource Broker+Information IndexCertification AuthorityUser InterfaceTESTBED SITE 1TESTBED SITE 2Job Status:Front EndCertificate ServerTESTBED SITE 3Graphics by Aldo StentellaGraphics by Aldo StentellaAfter creating a proxy process with temporary credentials (“grid-proxy-init” command), After creating a proxy process with temporary credentials (“grid-proxy-init” command), the user submits his/her job request to the Resource Broker.the user submits his/her job request to the Resource Broker.SubmittedSubmittedThe Resource Broker queries the Replica Catalog and Information Index to find suitable The Resource Broker queries the Replica Catalog and Information Index to find suitable resources (storage space, data and computing power).resources (storage space, data and computing power).WaitingWaiting InternetStorage ElementComputingElementWorker NodeUser InterfaceReplica Catalog+Replica ManagerResource Broker+Information IndexCertification AuthorityUser InterfaceTESTBED SITE 1TESTBED SITE 2Job Status:Front EndCertificate ServerTESTBED SITE 3Graphics by Aldo StentellaGraphics by Aldo StentellaThe Resource Broker selects the resources.The Resource Broker selects the resources.ReadyReadyThe Resource Broker submits the job to the selected Computing Element(s).The Resource Broker submits the job to the selected Computing Element(s).ScheduledScheduledThe Computing Element delivers the job to its Worker Node(s), which will do the real The Computing Element delivers the job to its Worker Node(s), which will do the real work.work.DoneDone InternetStorage ElementComputingElementWorker NodeUser InterfaceReplica Catalog+Replica ManagerResource Broker+Information IndexCertification AuthorityUser InterfaceTESTBED SITE 1TESTBED SITE 2Job Status:Front EndCertificate ServerTESTBED SITE 3Graphics by Aldo StentellaGraphics by Aldo StentellaOutput ReadyOutput ReadyThe output is moved to the Resource BrokerThe output is moved to the Resource Broker InternetStorage ElementComputingElementWorker NodeUser InterfaceReplica Catalog+Replica ManagerResource Broker+Information IndexCertification AuthorityUser InterfaceTESTBED SITE 1TESTBED SITE 2Job Status:Front EndCertificate ServerTESTBED SITE 3Graphics by Aldo StentellaGraphics by Aldo StentellaThe user gets his/her outputThe user gets his/her outputClearedCleared 网格计算环境示例GridWANClusterGridSMPGridSMPGridClusterUI/APIGridLANGridSMPGridSMPGridSMPGridClusterClusterClusterSMPGridClusterGridGridGridLANSite ASite BSite CUI/APIGridUI/APIGridLAN2021/3/938 结束语n虚拟天文台将构筑于网格环境之上n集群将构成网格环境中强有力的计算与服务支撑平台n并行计算将为具体事务处理提供技术与手段上支持2021/3/939 2021/3/940 放映结束 感谢各位的批评指导！ 谢谢 谢！谢！让我们共同进步2021/3/941 。
