Contacts

Department of Automated Programming

HPC & Distributed Computing Team

E-mail: dep145@gmail.com
Phone: +380445263603
Site: dep145.org.ua/en/
team leader: Vadim Tulchinsky

The team

The team is based on Automated Programming Department of Glushkov Institute of Cybernetics, National Academy of Science of Ukraine. The team stuff consists of 33 employees (researchers and programmers) and PhD students. 14 team members are candidates of science (~PhD), other have MS degree.

Objectives

The team has designed and developed a number of parallel, distributed data processing and computational applications, desktop, web, real-time programs, hardware/software complexes and ready-to-use systems. We constantly consider new challenging projects where the team can participate.

Skills

Languages: C/C++ (Microsoft/Intel/GNU), C#, Java, PHP, Perl, Bash, HTML, ActionScript, PowerShell, JavaScript, Fortran (Intel/GNU), MATLAB, VBA, XML, XQuery, SQL, QBE.
API
: MPI, OpenMP, OpenCL, CUDA, CUBLAS, CULA, Intel MKL, [SCA]LAPACK, FFTW, INPARLIB (own parallel math library), POSIX, MFC, COM, .NET, OpenGL, JSON, Lex & Yacc, OpenCV, OpenXML, iTextSharp.
DB: MS SQL Server, Oracle, MySQL, SAP HANA, MongoDB, MS Access, Micropoisk (own non-SQL database).
System
: Windows, Linux, HPC cluster architectures, assembly, support and administration.
Data processing: long experience of seismic data processing optimization (hundred gigs to terabytes per dataset), development of parallel and distributed data storages (Lustre, gridftp, dCache/SRM, Apache Hadoop, grid+cloud integration), machine learning, numerical methods and parallel algorithms.
Business intelligence
: e-business/e-government, digital signature and digital identification, product quality, etc.

International Collaboration

1-2 STCU projects (http://www.stcu.int/) each year since 1999 (current STCU projects: P516, P517); EU FP7 project SCUBE-ICT (2009-2010).

Experience. Selected projects

The team executes 5-7 projects each year. So here we put just samples of the developments selected by international or sign customers, long life or advanced technologies.

GeoPoisk geophysical program system for oil and gas industry

For Ukrainian State Geophysics Exploration Institute /UKRDGRI/ (Kiev, Ukraine) & Noyabrsk Oil & Gas Geophysics /NNGGF/ (Noyabrsk, Tyumen, Russian Federation)

Description: GeoPoisk (www.geopoisk.com, in Russian) is a software system for complex interpretation of geophysical exploration data from well log analysis to development of 3D geology models. It includes a lot of log method specific processing algorithms and presentation graphics, specialized high performance non-SQL database, C-style program language for multi-well batch processing, advanced reporting and querying tools, gridding, data cubes and sections, contour calculation in presence of geological faults, etc. Source data are remote compressed files in major industry formats. Service subdivisions of big oil companies (Gazprom, Gazpromneft, TNK, Lukoil, Surgutneftegaz), Russian subdivisions of Western service companies (Halliburton, Fugro, Baker-Huges, Schlumberger), local service providers, research institutes and universities of Russia, Ukraine and Uzbekistan are among GeoPoisk customers (~25% of the Russian market). The product is marketing since 1999.

Techniques: Visual C++, Lex & Yacc for the built-in interpretable scripting language, Micropoisk non-SQL database with SQL data source connectivity, supervised and unsupervised learning algorithms, dozen gridding methods including triangulation, splines, kriging, parametric statistical analysis, wavelet compression.

Forecasting ToolPack for Microsoft Excel

For Cyber-US, Inc. & Delta Airlines (Atlanta, USA)

Description: Forecasting ToolPak is a plug-in for time series analysis and prediction featuring model identification, simple and linear moving averages, exponential smoothing by Brown, Holt, Winters, etc.; ARMA-methods like Box-Jenkins, ARARMA, Generalized adaptive filtering, seasonal ARIMA; trend-based regression, S-curves (Gompertz, Logistic), linear and non-linear multiple regression, stepwise multiple regression; neural networks; seasonal decomposition like Census I, Census II (X11), Harrison’s harmonic smoothing (with the Fourier transform). The core functionality was later ported to Linux clusters under MPI (MPICH & LAM MPI) to reduce latency in the academic version of the product.

Techniques: C++, Microsoft Excel, VBA, methods of econometrics and statistics.

Duplex Wave Migration programs for seismic data processing

For Tetraseis Inc. (Calgary, Canada)

Description: Duplex Wave Migration /DWM/ (http://tetraseis.com/) is a patented (US 2006, UK, China) method of unconventional processing the conventional seismic exploration data. The method enables imaging of vertical and sub-vertical boundaries (faults, salt domes, and fracturing zones) which are not visible by conventional processing techniques. The migration belongs to the class of pre-stack depth migrations known by big datasets, intensive input/output and high (but linear) computational complexity. The program code initially implemented in 2006 for multi-core SMP Linux clusters has been step-by-step adapted to wide variety of storage and cluster architectures (including many-core/Intel Xeon Phi, GPU, MapReduce) and highly optimized on both theoretical and implementation levels. Tetraseis uses the product internally to execute service projects mostly on customer’s and rented clusters of thousands CPUs. Big oil/gas companies such as Lukoil, Conoco-Philips, BGP, Gazprom are among the service customers. The set of supported migrations is consecutively extended. Now it includes variations of DWM for VTI and TTI anisotropic media, conventional depth and time pre-stack migrations, post-processing tools. Saudi Aramco, Geoprime and Kelman Technologies run the programs on their clusters under our supervision.

Techniques: C++, MPI, OpenMP, POSIX threads, OpenCL, MapReduce.

Tesseral Pro, 2D and 2.5D full-wave seismic modeling

For Tesseral Technologies Inc. (Calgary, Canada)

Description: Tesseral Technologies (http://www.tesseral-geo.com/) is a leading provider of Windows desktop seismic modeling software. The series of projects targeted synthesis of 3D seismograms, which requires much more resources than for the 2D case. The projects included parallel implementation of 2D finite-difference simulation for multi-core servers, GPUs, Windows networks and Linux clusters, development of computational and memory efficient 3D modeling methods and their implementation for similar environments, visualization and animation. Due to synergy of memory decomposition scheme with GPU hardware the developed 2.5D Viscoelastic TTI Anisotropic method is a practical tool to solve complex 3D problems by a GPU equipped server or a small GPU cluster. Previously such problems were solved by supercomputers only. Conventional 3D techniques such as Ray Tracing and 3D Acoustic method have been implemented as well. The developed products are used worldwide by oil companies, advanced service providers, research institutions and universities for survey planning, design of test datasets, analysis of complex geology and training.

Techniques: C++, MPI, OpenMP, threads, OpenCL, CUDA, sockets, OpenGL.

Supercomputer Management System with job stream prediction

For RSC Group (Moscow, Russia)

Description: RSC Group (http://rscgroup.ru/en/) is a leading HPC hardware developer and integrator in Russia, developer of 50% Russian Top500 supercomputers. RSC ordered a commercial Supercomputer Management System on the base of SCMS open source solution developed by our team since 2006. The Supercomputer Management System key feature is usage of job stream prediction and history based application performance forecasting to improve load balancing, power consumption and resource allocation on multi-user clusters. E.g. for data intensive applications the system estimates performance vs. the number of CPUs, and so helps users to avoid input/output bottleneck. The system provides multi-channel automated hardware control of big systems for fault tolerance and non-stop service, flexible information security for both cluster and grid jobs, user-friendly GUI for end users and administrators, ready to use environments for popular scientific software (Gromacs, Gamess, NAMD, ABINIT, AMBER, Gaussian, etc.). The system contains automated balancing stands for the Linpack performance evaluation and the queue manager policy fitting.

Techniques: HTML, JavaScript, PHP, Apache, MySQL, IPMI, Linpack, Bash, sendmail, iptables, tftp, etc.

Grid storage of medical images for populational research and case studies

For Ukrainian National Grid /UNG/ (Ukraine)

Description: UNG (http://ung.in.ua/) is a grid community sharing resources of more than 30 HPC clusters located in academic institutes and universities of Ukraine for collaborative solution of big problems. UNG is also a member of European Grid Infrastructure (EGI) and expose resources for international projects. Our team develops and supports the biggest resource center of UNG since 2009 within a series of Ukrainian government funding projects. This project goal was to save big amount of depersonalized medical data so to make them available for statistical analysis from multiple hospitals and medical research organizations without access to personal information. At the same time the data of particular patient are supposed to be directly accessible by doctors in charge. The relation between database records of personal data and de-personalized image files (cardiograms in SCP-ECG format and tomography reconstructions in DICOM format) is established via random QR codes. The distributed grid storage has been assembled on the base of existed software. But the on-line data access and automated version synchronization needs have requested development of cloud in grid integrated solution.

Techniques: ARC/Nordugrid, dCache/SRM, LFC, Java, Apache, MySQL, JSON, XML, C/C++, PHP, Bash, remote desktop (VNC), virtual machines (xen, quemu), etc.

Management information system for bank analysts

For CREDIT EUROPE BANK (Ukraine)

Description: Management Information System (MIS) is a multi-layer decision support information system with reporting and analytical facilities. It was designed to support decision-making process in the bank administrative structure according the granted permissions and responsibilities. The information system layers are Front Office, Middle Office and Back Office. Front Office serves the bank local subdivisions which are providers of operational data. Its functions are input (web-based system), storing (relational database), reporting (HTML, TXT, XML). Middle Office for the bank regional divisions inputs and aggregates Front Office outputs of subordinated subdivisions, verifies the data (ETL-system), saves the aggregated and verified information in database, outputs simple analytical reports (MS Excel pivot tables). Back Office inputs aggregated information from Middle Office databases in top level of the information warehouse, performs analytical querying (OLAP) and reporting (MS Excel pivot table, web-based reports). MIS is an automated distributed system which satisfies various analytical needs of the bank central departments and divisions without engaging IT stuff.

Techniques: VBA, Java, MS Excel, MS SQL Server, Microsoft business intelligence stack.

Hardware/software complex of Certification Authority

For Ukraine Property Group Ltd. (Ukraine)

Description: The developed Certification Authority (CA) complex has passed the national accreditation in 2010 and became a working part of electronic document circulation in Ukraine. National Bank of Ukraine, Verkhovna Rada (parliament of Ukraine), Government of Ukraine are among its digital signature customers. According to Ukrainian legislation for creation of legal digital signatures CA must use a system of hardware certified according CWA (CEN/CENELEC) standards and certified software implementation of standard cryptography algorithms and digital signature formats. The system functions and features are regulated by a set of Ukrainian, European and international standards including DSTU 4145, ETSI TS 102 176, ISO/IEC 14888. The complex includes a test stand for automated verification of compatibility with other digital signature providers.

Techniques: Java, C++, C#, EJB, Linux, Windows, Oracle, cryptographic smart-cards, 3rd party software crypto-providers, digital signature & encryption algorithms.

Additional information

The team is experienced in English language support of international user community.


Vadim Tulchinsky (1968)

E-mail: dep145@gmail.com
Phone: +380445263603
head of department, project manager

Objectives

As head of department and project manager I represent a stable team of highly qualified HPC/HTC software developers, engineers and researchers. On behalf of the team I consider new challenging projects and international collaboration opportunities. We are experienced in product life cycle support from the stages of research and design to implementation, version control and user communications.

Skills

Management: international project management [12y], policy making [3y].
Languages: C/C++ [>20y], PHP [3y], HTML [5y], JavaScript [5y], Fortran [10y], VBA [5y], XML [2y], SQL [15y].
Parallel programing
: MPI [9y], OpenMP [4y], OpenCL [3y], CUDA [3y] , POSIX threads [5y] , MapReduce [1y].
DB: Oracle [4y], MS SQL Server [6y], Micropoisk (co-developer).
OS
: Windows, Linux.
Data processing
: seismic data processing [8y], machine learning, time series, gridding, compressive sensing, DSP & pattern recognition, numerical methods & libraries.

Education

1991 – software engineer (BC), Moscow Institute of Electronic Technology
1993 – engineer researcher (MS) in computer science, postgraduate courses of Moscow Institute of Electronic Technology
1995 – candidate of physic / mathematical sciences (PhD) in computer science, Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine

Occupation

Since 1993 to present time – from engineer to head of High Performance and Distributed Computations Lab and Automated Programming Department, Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine
2000 – senior developer, Telemate Software (Atlanta, USA)
1987-1993 – software developer and post graduate student, Moscow Institute of Electronic Technology

Research

Parallel and distributed computing, grid, cloud; big data, seismic data processing, databases; simulation, computational methods in geophysics. 65 publications.
Senior researcher (2002), EAGE member (2008), SEG active member (2009).
Ukrainian President's Prize for Young Researchers (2001), Award of the NAS of Ukraine named after S. Lebedev (2013).

Experience. Selected projects

Project

Responsibility

Duration

GeoPoisk geophysical program system for oil and gas industry (www.geopoisk.com)

project manager

Since 1999

Tesseral Pro, 2D and 2.5D full-wave seismic modeling (www.tesseral-geo.com)

project manager

Since 2001

Series of intelligent parallel workstations “Inparcom” for State R&P Enterprise “Electronmash”

system software team leader

Since 2004

Duplex Wave Migration programs for seismic data processing (tetraseis.com)

development team leader

Since 2006

EU FP7 SCUBE-ICT is to upgrade strategic ICT R&D cooperation between the EU, Belarus and Ukraine (www.scube-ict.eu)

team leader

2009-2010

Resource center of Ukrainian National Grid  (funded by the state R&D programme of grid technologies)

team leader

2009-2013

Development of software design, concepts and architecture of the integrated data-processing system “Electronic Parliament of Ukraine”

team leader

2011-2012

Development of energy efficient supercomputer (co-funded by RSC Group /Moscow, Russia/ & State Agency on Science, Innovations and Informatization of Ukraine)

team leader

2012

Scientific and technical expertise of the National Informatization Programme tasks and projects for State Agency on Science, Innovations and Informatization of Ukraine

expert team leader

2011-2013

Additional information

I’m a member of Coordination Committee of Ukrainian National Grid (since 2010).

 

Department Projects

International projects (2010-2015)

  • 2012-2015: Natural Resource Canada ecoENERGY Innovation Initiative
    Applying Duplex Wave Migration (DWM) and Seismic Modelling Technology to Delineate Fractures within Cap Rock over Enhanced Oil Recovery sites for Mitigation of Potential Economic and Environmental risks and reducing CO2 emissions
  • 2012-present time: STCU P516
    Development of Parallel Algorithms and Programs for 2D and 3D Seismic Modeling, Synthetic Seismogram Visualization and Processing on the base of Tesseral Software Package
  • 2012-present time: STCU P517
    Development of Theory and Programs for Attribute Analysis basing on Duplex Wave Migration for Obtaining more Detailed Model of Carbonate Oil and Gas Reservoirs
  • 2014: SAP PartnerEdge HANA/GeoPoisk
    Application of SAP HANA in-memory database in remote cloud for  acceleration of geophysical data processing
  • 2007-2011: STCU P-206(A, B, C)
    Development on the base of Tesseral software the 2D and 3D duplex wave migration theory, algorithms and a program for Linux cluster
  • 2009-2010: FP7 SCUBE-ICT
    Strategic Cooperation in Ukraine, Belarus and EU in Information and Communication Technologies

Local projects (2010-2015)

1. State R&D Programme “Development and Implementation of Grid Technologies for 2009-2013 years” (SP-Grid), National Academy of Science of Ukraine Programme “Grid Infrastructure and Grid Technologies for Science and Applied Research” (AP-Grid)

  • 2009-present time: SP-Grid, AP-Grid
    Development of National Grid Resource Centre at the Glushkov Institute of Cybernetics of NASU
  • 2010-2013: SP-Grid
    Intellectualization of cluster computing information technologies in grid environment
  • 2010-2013: SP-Grid
    Development of algorithms and software programs to adapt 3D finite-difference modeling and migration transformation to grid environment
  • 2012-2013: SP-Grid
    Development algorithms and software to adapt grid environment the programs to study radiation-stimulated processes and phase transformations isomorphic substitutions in mineral structures
  • 2013: AP-Grid
    Energy Conservation Complex of Resource Centre of Ukrainian National Grid.
  • 2011: SP-Grid
    Development of methods and tools of grid-technology information security based on international standards

2. National Academy of Science of Ukraine Programme “Development of effective intellectual information technologies, high productive computers and information security tools (Intellect)”.

  • 2015: Intellect VK.145.23
    Development of power saving infrastructure of supercomputer complex SCIT
  • 2014: Intellect VK.145.22
    Development of cloud infrastructure to improve applicability of supercomputer complex SCIT
  • 2013: Intellect VK.145.19
    Development of high-performance environment of multiple petabytes data storage to SCIT supercomputer complex of the Glushkov Institute of Cybernetics of NASU
  • 2012: Intellect VK.145.18
    Strengthening the national cluster complex SCIT-GPU and development of IT management tools for heterogeneous cluster systems
  • 2011: Intellect VK.145.13
    Development of national cluster complex SCIT-GPU and improve the intelligent information technologies for solving problems of extra big complexity
  • 2010: Intellect VK.145.12
    Development of technical capacity of national cluster complex SCIT and intelligent information technologies for solving problems of extra big complexity

3. Other HPC/HTC related projects

  • 2012: State Agency of Innovations + RSC Technology (50% x 50%)
    Development of national energy efficient supercomputer for solving complex scientific and technical problems and problems of public administration
  • 2012: National Academy of Science of Ukraine, VK145.14
    To develop models and methods of extra big data warehousing with advanced content analysis
  • 2011-2013: National Academy of Science of Ukraine, VK146.02
    Development of methods and tools for high performance computing in distributed heterogeneous environment
  • 2007-2011: National Academy of Science of Ukraine, VK105.02
    To develop methods of intelligent IT for optimization and verification by deductive method of parallel programs which are scaled up
  • 2009-2010: TetraSeis Inc. (Canada)
    Analysis of computation speed and optimization of parallel programs for seismic data processing