Charles Thayer (Not Looking)
thayer@b2si.com, San Francisco, +1.415.648.1456, Cell: +1.415.378.5924

Summary:

    I'm a Unix systems programmer who has been working on large distributed data systems (3,000+ nodes) and cluster management in C++. Currently building a next-generation multi-tenant HBase/HDFS/Hadoop cluster application in Java which I was the lead architect/designer for.

    Objective: To find a position as a senior engineer for a company building industrial-strength, back-end systems, where I can solve problems that require scalable, high performance, and robust solutions.

    Experience: Designing scalable software systems and architectures; Building products, technology, and infrastructure; And deploying and maintaining online services.

    Potenial Projects: Adding online cluster expansion/contraction features to HDFS and HBase; Creating new schedulers for Hadoop clusters; Building cluster management tools; Designing multi-cluster resource managers/brokers.

 

Technical Skills:


Languages: C/C++, Python, Perl, Java. Plus: PHP, Javascript, Tcl/Tk, Unix Shell (Bash, KSH), and others.
Databases: SQL (MySql, Oracle), Database and Schema Design, Oracle PL/SQL, Postgres, Sqlite.
Systems Programming: Linux (Ubuntu, RedHat), UNIX (Solaris, SunOS, HPUX).
Networking: TCP/IP, UDP, HTTP, SSL, DNS/Bind, IRC/Jabber, LDAP, YP/NIS, NFS, BOOTP/TFTP/DHCP, FTP, LPD, Inetd, Tcpd, Amd, etc.
Technologies: HBase/HDFS/Hadoop, OOP & OOD (Object-Oriented Programming and Design), Asynchronous programming / Threads, SGML/HTML/XML/XMLRPC. Plus: UML, Apache Modules, PDF/PS/LZW, Some Linux Kernels & Device Drivers.

Work Experience:


  • Design: Plan and develop company-wide software libraries, APIs, protocols, databases, and other infrastructure.
  • Review: Conduct code reviews, system design reviews, performance analysis, technical mentoring.
  • Test: Design and build unit tests, system tests, and staging / roll-out environments.
  • Lead: Evaluate feasibility and suitability of designs and technologies, given business and resource constraints.
  • Run: Maintain and operate online services with e-commerce transactions 24x7.

 

Work Experience Highlights:


    4/2003 to present (7 yr)
    Senior Engineer
    Yahoo Inc., 701 First Ave., Sunnyvale, CA

    Worked on the "Web Search Content System Team" on cluster management and monitoring software (C++), distributed data storage, and the operation of the production web crawlers. During this time the main web crawl system started at 100 nodes, went through 5 cluster doublings, and is currently 3,600 nodes.

    • Content Operations Team (1yr): Managed the deployment, operations, and monitoring of production web crawler clusters in distributed data centers. Typical projects included moving 1.8 PB across coasts. Catalog Agility Project: Primary architect on crawler scalability and data storage refactoring and changes to sharding (C++) resulting in a "You Rock Award".
    • Crawler Team (4.5 yr): Catalog Master: Built and extended cluster management and monitoring software for managing thousands of nodes (C++). Many projects ranging from investigating problems based on statistical anomolies in freshness, comprehensiveness, and quality (Python) to writing disk burnin software (C) used by all of YST (Yahoo Search Technology). Wrote CTA package (Cluster Trends Analyzer) in Python and CluStats (Cluster Stats and alerts) libraries in C++.
    • Echelon Team (1.5 yr): Echelon Master: worked on cluster management software for controlling indexing pipeline and batch processing (C++). Echelon Copier (EchCp in C & C++): cross-coast database distribution software with extensive performance throttling controls for delivering production index data to other groups.

    Technologies: Unix Systems Programming; Networking; C++, Python, Perl, Shell, SQL.

    Yahoo logo

    6/2001 to 3/2003 (1 yr, 9 mon)
    Senior Engineer
    Tapstone Inc., Walnut Creek, CA. (Start-Up, 12 developers, 20 people)

    Tapstone's Server Change Center is an IT tool for managing enterprises with hundreds to thousands of machines. Agent software running on each computer detects changes to the file system, the registry, and configuration files (on both Solaris and Windows.) These agents each communicate a stream of events in real-time to a central appliance (running Linux and Postgres.)

    • Designed the distributed network agent for portability, scalability, and low-impact.
    • Wrote low-level back-end software for Solaris, Linux, and Windows NT in Python and C++. Implemented hardware scanner, control daemon, and various libraries.
    • Reviewed code, unit tests, system tests, and design work as the manager and lead engineer of the agent group (4-6 developers, including myself.)
    • Designed APIs, libraries, and protocols as lead engineer of the architecture team during system design phases. Reviewed system-wide design and implementation choices to support the goals of high performance, fail-safe operation.
    • One of the authors of the provisional patent application: Management and Monitoring System for Distributed Computer Networks.

    Technologies: Systems Programming (cross-platform); Python, C++, Java; TCP/IP, XMLRPC, SQL.

    Tapstone logo

    3/2000 to 6/2001 (1 yr, 3 mon)
    Chief Technology Officer (CTO/2)
    CityRealty.com Inc., NYC (Dot-com, 8 developers, 16 people)

    Built the technology for "The New York City Real Estate Exchange." CityRealty.com was a small on-going venture with new funding, from Goldman Sachs, to create sophisticated web personalization services for residential real-estate brokers, buyers, renters, and customer service representatives. Shared the role of CTO with partner from Mediabridge Infosystems Inc.

    • Designed the software systems and databases behind the website.
    • Built the calendaring and scheduling applications. Utilized web and email interfaces with an Oracle back-end to track and coordinate face-to-face meetings and resolve conflicts.
    • Designed and developed core PHP libraries for interfacing to the Oracle database. Implemented various core libraries for personalization and saved searches.
    • Designed and implemented software roll-out process from development, to QA, to production servers.

    Technologies: PHP, Apache; Oracle; Solaris, Linux.

    CityRealty logo

    10/1994 to 3/2000 (5 yr, 5 mon)
    Chief Scientist, Founder
    Mediabridge Infosystems Inc., NYC
    (Consulting Firm, staff of 4-6 plus sub-contractors)

    Built a small firm that provided Internet technology and consulting to businesses based on the Web in New York City. The following projects reflect a sample of the technical work designed and built by myself for the company:

    • Metrobeat: Became Citysearch (now part of TicketMaster). Implemented system that published the 40,000 page website daily using a dozen templates and a large Oracle database. Designed and wrote the template engine in C and the apache module which connected it to Oracle.
    • DDB Needham: Advertising Agency. Built intranet platform and online library for world-wide management of creative and multimedia content used in pitches and campaigns. Mainly Perl.
    • ReturnMe.com: E-Commerce site using Cybercash, SSL, Perl CGI, PHP.
    • Bartleby.com: Huge site generated from SGML/XML from multiple vendors in Perl with complex SGML DTDs.
    • Morgan Stanley: ClientLink 2.0; WSRAPI 1.0 & 2.0; Coverage Tracker; CMBS PDF Reporter; Built with Perl, Sybase, C/C++.
    • J.P. Morgan: Virtual Librarian Project for managing online training tools. Perl, Java Script.
    Mediabridge Infosystems logo

Education:


1987-1991: Columbia University, School of Engineering and Applied Science

    Bachelors of Science in Electrical Engineering (BSEE).

    Masters EE: Worked toward a masters in EE with a focus on VLSI design, while employed by the Computer Science Department at Columbia University. Left to become CEO of a start-up in 1994 without completing degree.

    Teaching Assistant: Object-Oriented Design and C++ (John Lakos), Networks (Matt Blaze), C-Programming (Dan Duchamp), Scientific Computation (Joseph Traub).

    Highlights: Worked with John Lakos author of "Large Scale C++ Software Design" when C++ was first being taught at Columbia. Got to meet Donald Knuth (and read his .emacs). Worked under Alfred Aho (who still prefers his awk to GNU's gawk). Saw Bjarne Stroustrup's hillarious April Fool's Day lecture at NYU (and drank beer with him after.) Had Mischa Schwartz as my undergraduate advisor. Invented a modular connector for a communication line.

Hobbies: Gardening, Electronics/Circuits, Woodworking.

Recommendations: Available upon request. Last Revised $Date: 2011/02/24 22:13:43 $