API Builder Tools 4.0.0 Release Note

API Builder 4.0.0 - 29 June 2018

Summary

API Builder V4.0 enables customers to install and run API Builder in containerized environments. Previously, API Builder could only be installed in a cloud environment. For information on installing and getting started with API Builder V4.0, refer to the API Builder Getting Started Guide. The API Builder 4.0.0 release includes the following changes and new features to implement the ability to install API Builder on-premise. The release also includes breaking changes, fixed issues, known issues, and security vulnerabilities.

This release includes:

Upgrade 

For more detailed instructors on how to migrate to API Builder 4.0.0, please follow the API Builder v3 to v4 Upgrade Guide.

Breaking Changes

  • Node Handlers from API Builder v3 are no longer compatible with API Builder v4. Node Handlers are now referred to as Flow-Nodes.
  • API Builder no longer loads Node Handlers from the ./nodehandlers directory.
  • Existing Node Handlers are renamed and installed as Flow-Node plugins. More information can be found here.
  • The Admin UI has been removed from the API Builder runtime and should be installed as an explicit development dependency: npm install --save-dev @axway/api-builder-admin@^1.0.0
  • The Appc CLI is no longer used to initialize, run or deploy API Builder v4 services. @axway/api-builder is the new CLI which should be used to create new projects. appc run is no longer required, replaced with directly starting your service with node app.js. npm install is now explicitly required before API Builder can start in order to pull the required dependencies from NPM.
  • The minimum required version of NodeJS is now 8.9.
  • The prefix of Model Flow-Node IDs has changed from nodehandler://arrow-flow-invoke to nodehandler://api-builder-flow-invoke.
  • Previously, without logLevel defined in the configuration, the default level would be 'trace', now it is 'none'. New projects are generated with logLevel defined as 'debug'.
  • Previously, API Builder used different API keys based on the environment in which the service was running. These were accessible in default.js as apikey_development, apikey_production and potentially apikey_preproduction. Now, a single value apikey is used instead. If the value still needs to be changed per-environment this value should be configured using an environment variable instead.
  • Previously, a number of appc_ prefixed variables were available in API Builder Web, used for accessing environment and platform-specific information such as the environment and host. Now, these variables have been removed and should be accessed in an alternative way.
  • The /arrowPing.json endpoint has been renamed to /apibuilderPing.json. (Note that this has been re-added in the Barcelona release).
  • Previously, configuring the API Builder port by setting it to 0 would let the service start on any available port. Now, the feature has been removed and the configured port must be valid and available.
  • Previously, in config, baseurl could include the port as well as the hostname. Now, it is more strict and should not include the port.
  • RDPP-4075: Previously, API Builder loaded Service Connectors from the ./serviceconnectors directory. Now, they are no longer supported.
  • RDPP-4390: Previously, API Builder loaded Data Connectors from the ./connectors folder of your project or the ./node_modules/connectors folder. Now, API Builder loads Connectors as API Builder plugins which should be installed as npm package dependencies and accessible from ./node_modules.
  • RDPP-4390: Data connectors from API Builder v3 are no longer compatible with API Builder v4. The Appc CLI is no longer used to install connectors. For a list of available Data Connectors in API Builder v4 and how to use them, refer to API Builder Connectors.
  • RDPP-4225: Previously, it was possible to specify environment variables from the operating system environment and apply them to the API Builder runtime (for example, ARROW_PORT). Now, it is no longer possible to automatically use ARROW_* environment properties. Instead, the configurable environment properties need to be explicitly included in the API Builder's configuration files (for example, conf/default.js), and reference the environment variable by name (for example, process.env.APP_PORT).
  • RDPP-4225: Previously, it was possible to have multiple sets of configuration files, for example, "development", "production", and so forth. Now, only two sets of configuration files are recognized: "default" and "local" (for example, default.jsfoo.default.js, local.js, and foo.local.js), where the local config files are ignored by Git and NPM.
  • RDPP-4348: Previously, API Builder would write aggregated transactional logs to ./logs. Now, API Builder logs everything to the console and as a result, the logging in conf/default.js is no longer used. The existing logging configuration will have no effect on the application and can be removed safely.
  • RDPP-4390: Previously, Data Connector configuration would use the connector name as the key; for example, appc.mysql, and then list specific configuration properties for the connector, including an optional alias. Now, the Data Connector configuration will use the user-configurable alias as a key; for example, mysql. The property connector is now required and must be set to the package name of the Data Connector being used.

    mysql: {
    	connector: '@axway/api-builder-plugin-dc-mysql',
    	user: 'root',
    	password: 'root'
    }
  • RDPP-4577: Previously, the Swagger generated for model CRUD APIs accepted additional properties in the request. This could lead to errors and unexpected behavior. Now, the APIs do not accept additional properties and they precisely define their return type in the id field.
  • RDPP-4781: Previously, the API Builder runtime was in a module called arrow and was referred to as Arrow. Now, the API Builder runtime is in a module called @axway/api-builder-runtime and is referred to as APIBuilder.
  • RDPP-4781: Previously, the API Builder React engine was in a module named arrow-react-engine. Now, the API Builder React engine is in a module called @axway/api-builder-react-engine.
  • RDPP-4152: Previously, the methods on the model flow-nodes outputted Arrow.Model objects to the Flow context. Interacting with these in subsequent flow-nodes required a knowledge of the internal workings of the Model object. Now, the model flow-nodes output plain data objects containing the fields and values from the model method. This will only cause issues for flows that were expecting the Model object rather than a data object.

  • RDPP-4368: Previously, the distinct method on flow-nodes for connectors had parameters for sel and unsel that were ignored. Now, these parameters are removed from the model flow-node.
  • RDPP-4550: Previously, endpoints with query parameters accepted page and per_page as well as skip and limit. The problem is that it wasn't clear how to effectively use these. To exacerbate the issue, supplying both sets of parameters would yield unexpected result ranges. Now, page and per_page parameters have been removed since their functionality is a subset of what can be provided by the skip and limit parameters. In the backend, page and per_page are computed via skip and limit so connectors that rely on them should continue to function.
  • RDPP-4602: Previously, model fields with default values would get set in the model schema without validation or interpretation. Now, the default values are validated to ensure that they are cast as the correct type.
  • RDPP-4712: Previously, models would auto-generated API with optional body parameters for the CreateUpsertUpdate, and Find and Modify methods. Now, body parameters are required.
  • RDPP-4732: Previously, there was a serialization.exposePrimaryKeyAsId configuration option that was intended to force the primary key of a model to always be called id. Now, this configuration option has been removed and the model primary key field will always be exposed using its actual column name.

Features 

  • RDPP-1243: Previously, flows had no inputs and would receive implicit runtime parameters $.params$.request$.config, and $.env for use at runtime. Now, those parameters are explicitly part of the flow definition.

  • RDPP-1243: Previously, "Generate endpoints" from a model would generate Flows that had implicit $.params$.request$.config, and $.env inputs for use at runtime. Now, the generated Flows only use the explicit API endpoint parameters needed to execute the model function, e.g. $.limit and $.where.
  • RDPP-4045: Previously, the flow editor UI had a Save button that would save the current flow being edited and then exit to the list of API methods. Now, the flow editor UI has an Apply button that allows the flow to be saved, but stays in the flow editor and does not exit. The Cancel button has also been renamed to Close. If the flow has no changes, then Close will just exit. If the flow has changed, then Close will prompt to save or discard changes before exiting.
  • RDPP-4346: Previously, projects created with the Axway Flow SDK had to synchronously export their flow-node specifications which prevented the use of asynchronous APIs, such as network calls or complex parsers, when generating the flow-node specifications. Now, API Builder allows the flow-node projects to export a Promise which will resolve with the flow-node specification, allowing for asynchronous loading of the flow-node specifications. Now, this is the default behavior in all new flow-node projects.
  • RDPP-4392: Previously, the appc.composite Connector was installed separately into your application. Now, the Connector has been renamed to composite and is now included as part of API Builder with no need to install it separately.
  • RDPP-4524: Previously, when a Flow was invoked it would log the step-by-step execution with only the Flow-Node's ID which was not very user-friendly. Now, Flow logging will include the user-provided Flow-Node name alongside the ID. For example, Format String (mustache.1).

Fixes 

  • RDPP-4350: Previously, importing API endpoints into API Builder would incorrectly apply a Content-Type to GET methods. Now, imported endpoints do not apply Content-Type to GET methods.
  • RDPP-4350: Previously, API Builder would apply consumesproducestagsschemes, and security properties on the service's Swagger document. This would unintentionally mean that any endpoints without these properties defined per-path, would take these global values instead of their intended value. Now, any endpoints which are imported with any of these properties will only have them applied to that endpoint's methods, and API Builder will not apply any of these properties globally which could change the meaning of the resulting Swagger document.
  • RDPP-4355: Previously, HTTP method verbs in the API Documentation page for endpoints were lower-case. Now, they are correctly displayed in upper-case.
  • RDPP-4363: Previously, importing a Swagger document with an empty title would save the file to endpoints/*.json. This is because the file name is derived from the title. Also, if a file already existed with the desired name, an error would be thrown. Now, the imported file name or URL basename will be used if the Swagger title is blank. Additionally, if the desired filename already exists, then the imported filename will have an integer appended to it to ensure uniqueness.
  • RDPP-4368: Previously, the model generated API for distinct had options for sel and unsel that were ignored. Now, these options are not added to the API.
  • RDPP-4531: Previously, query parameters in the Swagger generated from models were missing some maximum, minimum, and default properties. These are used to set parameter defaults and to enforce correct user input. For example, the page query parameter should be set to a minimum value of 1. Now, these values are part of the generated query parameters.
  • RDPP-4559: Previously, when running an API Builder project that began with the character "u" on Microsoft Windows, the service would fail to start with a SyntaxError: Invalid Unicode escape sequence. Now, these projects will start correctly.
  • RDPP-4578: Previously, after importing a Swagger API, when trying to invoke an API method that consumes a body, where the consumes parameter was not defined in the original Swagger document (either on the method, or globally), API Builder would return "Request validation failed: Parameter (body) failed schema validation". Now, API Builder applies a default consumes parameter of application/json. The Swagger 2.0 specification does not explicitly handle this condition but their own tooling applies application/json and it is a reasonable assumption that the method can default to application/json when the body must validate against a JSON schema.
  • RDPP-4732: Previously, the primary keys on models had to be named id. Now, API Builder supports primary key columns with names other than id.

Known Issues 

  • RDPP-3825: Filtering the API Builder Console administrator access using IPv6 addresses may cause ENOTFOUND errors.
  • RDPP-3867: When attempting to create and save a flow for an imported Swagger endpoint that contains a path or paths defined by references the save will fail.
  • RDPP-3960: The API Builder Console does not recognize a required consumes value for form parameters if it is appended and the endpoint load will fail. For example:

    "consumes": [
    "multipart/form-data; charset=utf-8"
    ],

    The appended character set (charset=utf-8) will cause the endpoint load to fail.

  • RDPP-3979: When deleting endpoints which contain references within paths, a Page Not Found (404) error may be displayed in the API Builder Console. For example, a Swagger document with references within paths may look similar to this:

    {
    	"swagger": "2.0",
    	"paths" {
    		"x-path": {
    			"get": {}
    		},
    		"/find": {
    			"$ref": "#/paths/x-path"
    		},
    		"/search": {
    			"$ref": "#/paths/x-path"
    		}
    
    	}
    }

    The API Builder Console will fail to find GET /find since it is inside a $ref. If the API Builder Console has modified the referenced $ref, it could cause unexpected behavior for other paths referencing #/paths/x-path such as /search - deleting GET /find could unexpectedly delete GET /search too. 

  • RDPP-4050: When rendering the flow editor, the API Builder Console may fail to render the Scalable Vector Graphics (SVG) icons correctly in the Firefox browser. The render failure may result in blank icons being displayed in the tool panel and in the flow diagram. This is due to a long-standing bug in Firefox that fails to scale SVG graphics correctly. To fix, edit the SVG icon and add height and width. For example: <svg ... height="80" width="80" />
  • RDPP-4280: Editing large object parameters on the API Orchestration page in the API Builder Console may cause multiple, confusing flow-node configuration panel scrollbars to appear.
  • RDPP-4445: If a Swagger definition uses the allOf parameter, the body generated in the method test window is incorrect.
  • RDPP-4532: Given a data connector that generates models from a database, and is configured to auto-generate the API for those models (modelAutogen is set to true), and there exists a table with a primary key that is not an auto-incremented number and not named "id" (for example, is a PK on username), then the API generated will not properly handle the methods for Upsert or Update. In the UI, Upsert will render an example body with the non-id PK (for example, username) and id and will also fail 500 with various error messages depending on content of the body, "You must provide a Model id and data Object, that will be persisted", required body parameter: username missing", "You must provide a Model id and data Object, that will be persisted". Update will fail with 500 error "invalid field: id". Given a table "geozip", postalCode (PK string), lat (string), lng (string), the generated API methods disallows Create or Upsert.
  • RDPP-4716: Given a MySQL data connector that generates models from a database, and is configured to auto-generate the API for those models (modelAutogen is set to true), and there exists a table with no primary key, then the API Builder will not be able to handle the following methods: UpdateDeleteFind By IDFind and Modify, and Upsert.
  • RDPP-4724: Given an Oracle data connector that generates models from a database, and is configured to auto-generate the API for those models (modelAutogen is set to true), and there exists a table with no primary key, then the API Builder will not be able to handle the following methods: UpdateDeleteFind By IDFind and Modify, and Upsert.
  • RDPP-4727: When executing a count query on a composite model that is being sourced from an Oracle DB Model the reported result count will never exceed 10 irrespective of how many actual rows there are.
  • RDPP-4736: Given a Swagger with an extension, for example, on the paths object, the Swagger flow-node plugin can fail to load the Swagger file, resulting in an error: "Error loading plugin: @axway/api-builder-plugin-fn-swagger. Cannot convert undefined or null to object."
  • RDPP-4748: The Swagger document describes the application's configured schemehost, and port; however, the application's schemehost, and port are not editable via the application's configuration.
  • RDPP-4752: The format of a distinct query's response depends on the type of the connector.
  • RDPP-4759: Calling Update or Find and Modify on a Model that uses the composite connector and contains the required fields may fail and cause the server to terminate.
  • RDPP-4795: The Mongo database connector does not work well with public keys (PKs) that are not ObjectID.
  • RDPP-4813: If the endpoint Swagger file in the  /endpoints directory contains special characters in its name, for example [test].json, the endpoint is not rendered correctly in the UI.

Security Vulnerabilities 

  • RDPP-4810: JSONSelect

    • Vulnerability: JSONSelect@0.4.0 is used indirectly by the api-builder-runtime. JSON path is used in the flow engine and flow editor to query object value(s) at runtime. The issue identified by the CVE is with a jQuery artifact that is vulnerable to XSS that is bundled with JSONSelect. jQuery version 1.6.1 has the following relevant CVEs reported against it:

    • Analysis: The JSONSelect npm package includes in its distribution, a copy of jQuery that is used for unit-test and is not used by JSONSelect itself, and as a result, it is not a risk for API Builder.
  • RDPP-4811: Bootstrap 20184

    • Vulnerability: API Builder UI uses react-bootstrap which has a dependency on Bootstrap v3.3.7, and it is 3.3.7 that contains the vulnerability. At the time of writing this ticket, Bootstrap did not publish a 3.x version after 3.3.7, so no fix was ever released. The Bootstrap library is now at 4.x. However, the react-bootstrap library that API Builder uses is actively working on a version compatible with Bootstrap v4.x. The following issue is logged against Bootstrap v3.3.7:
      • 20184 - XSS in data-target attribute
    • Analysis: The vulnerability and risk are documented in 20184. The API Builder UI only runs on the developer machine and is locked to localhost by default. The API Builder UI will not be installed in production. Furthermore, the UI bundled with API Builder does not use data-target attributes. The risk to API Builder is low.

Notable Connector Fixes

MySQL

  • RDPP-4368: Previously, calling query on the MySQL connector with sel containing an unknown field would have indeterminate results and would be potentially dangerous for SQL injection attacks. Now, unknown fields are filtered out and will return an error.
  • RDPP-4368: Previously, calling query on the MySQL connector with order containing unknown columns would have indeterminate results and was open to SQL injection attacks. Now, using order with an unknown column will return an error.
  • RDPP-4368: Previously, calling distinct on the MySQL connector with an unknown field would have indeterminate results and would be potentially dangerous for SQL injection attacks. Now, an unknown field will return an error.
  • RDPP-4368: Previously, calling querydistinctcount, or findAll on the MySQL connector with a where option that contained unknown columns would have indeterminate results and was open to SQL injection attacks. Now, using the where option with an unknown column will return an error.
  • RDPP-4368: Previously, calling distinct or findAll on the MySQL connector would not honor options for orderskip, or limit. Now, these options are honored.
  • RDPP-4368: Previously, calling upsert on the MySQL connector with an un-escaped ID value would have indeterminate results and would be potentially dangerous for SQL injection attacks. Now, ID is passed as a placeholder and escaped.

  • RDPP-4548: Previously, the MySQL connector did not support the decimal type. Now, support has been added for decimal table fields and converts them to a number in the corresponding JavaScript model.

Mongo

  • RDPP-4700: Previously, the distinct method for Mongo database collections did not honor skiplimit, or order query parameters. Now, the distinct method for Mongo database collections honors skiplimit, or order query parameters.

Related Links