OpenAPI and Swagger (API First Development)

The Swagger tools, and the OpenAPI format, are an excellent way to document REST API's and even to generate client or server stub libraries to ease implementation. The technology serves two purposes 

                a) standardized documentation for REST API's,

                b) generating code from API documentation in several programming languages.

An OpenAPI file is fairly simple to write, An OpenAPI file allows you to describe your entire API, including:

• Available endpoints (/users) and operations on each endpoint (GET /users, POST /users)
• Operation parameters Input and output for each operation
• Authentication methods   
• Contact information, license, terms of use and other information.

API specifications can be written in YAML or JSON..

What are they?

https://swagger.io/blog/api-design/openapi-driven-api-design/

OpenAPI = Specification (OAS Open API Specification)

Swagger = Tools for implementing the specification

The development of the specification is fostered by the OpenAPI Initiative, which involves more the 30 organizations from different areas of the tech world — including Microsoft, Google, IBM, and CapitalOne. Smartbear Software, which is the company that leads the development of the Swagger tools, is also a member of the OpenAPI Initiative, helping lead the evolution of the specification.

Swagger tools which can be used at different stages of the API lifecycle

Swagger Editor: Swagger Editor lets you edit OpenAPI specifications in YAML inside your browser and to preview documentations in real time.

Swagger UI: Swagger UI is a collection of HTML, Javascript, and CSS assets that dynamically generate beautiful documentation from an OAS-compliant API.

Swagger Codegen: Allows generation of API client libraries (SDK generation), server stubs and documentation automatically given an OpenAPI Spec.

Swagger Parser: Standalone library for parsing OpenAPI definitions from Java

Swagger Core: Java-related libraries for creating, consuming, and working with OpenAPI definitions

Swagger Inspector (free): API testing tool that lets you validate your APIs & generate OpenAPI definitions from an existing API

SwaggerHub (free and commercial): API design and documentation, built for teams working with OpenAPI.

Eclipse tool to develop swagger:

KaiZen OpenAPI Editor is an Eclipse editor for the industry standard API description language, formerly known as Swagger. It now supports both Swagger-OpenAPI version 2.0 and OpenAPI version 3.0.

Basic Structure of an OpenAPI Specifications file:

Specification documentation:

https://swagger.io/docs/specification/about/

All keywords are case sensitives.

• -          Metadata
• -          Servers
• -          Path
• -          Parameters
• -          Request body
• -          Responses
• -          Input and output models
• -          Authentication

Example:

Generating server-side code:

Swagger-codegen:

                Download the code and build jar or download jar directly from below url

https://github.com/swagger-api/swagger-codegen

https://github.com/swagger-api/swagger-codegen/wiki/server-stub-generator-howto

                java -jar swagger-codegen-cli-3.0.0.jar generate -i api\openapi.yaml -l spring --library spring-mvc -o api\mvc -c api\option.json

JHipster codegen:

Generate a JHipster application with OpenApi Enable. You can configure api and model package

<configuration>

                <inputSpec>${project.basedir}/src/main/resources/swagger/api.yml</inputSpec>

                <generatorName>spring</generatorName>

                <apiPackage>com.first.services.web.rest</apiPackage>

                <modelPackage>com.first.services.web.model</modelPackage>

                <supportingFilesToGenerate>ApiUtil.java</supportingFilesToGenerate>

                <configOptions>

                                <delegatePattern>true</delegatePattern>

                </configOptions>

</configuration>

Write your specification using a tool such as swagger-editor, put it in src/main/resources/swagger/api.yml, then run

./mvnw generate-sources

or

./gradlew openApiGenerate

You can move your generated classes to your src/main/java or keep them in target directory ( target directory classes are in classpath)

After moving the classes to src/main/java rename ApiUtil class to a meaningful name with keeping the reference.

Then implement the “Delegate” interfaces generated in ${buildDirectory}/generated-sources/openapi/src/main/java/${package}/web/api/ with @Service classes.

Change the @RequestMapping("/api") annotation located in com.first.services.web.rest

Create JDL file to create entity object

jhipster import-jdl {fileName}

Map the delegate interface implementations to entity object.

  

Apache CXF creating rest client authentication and customize header

There are two ways you can authenticate rest client.

Setting header authorization:

WebClient client = WebClient.create(url);

String authorizationHeader = "Basic " + org.apache.cxf.common.util.Base64Utility

                                    .encode((username + ":" +  password)).getBytes());

                    client.header("Authorization", authorizationHeader);

Pass user and password when you create your rest client.


WebClient client = WebClient.create(url, username, password, null);



If you want to pass client_id or any other parameter just use

client.header("CLIENT_ID",{your_id})

Settings media type:

client.type(MediaType.APPLICATION_JSON_TYPE)

client.accept(MediaType.APPLICATION_JSON_TYPE)

Then follow the url to get the response from server.

https://problemslicer.blogspot.com/2018/09/no-message-body-reader-has-been-found.html

How to clear local working directory (untracked) all manually added files

To reset a specific file to the last-committed state (to discard uncommitted changes in a specific file):
git checkout thefiletoreset.txt
To reset the entire repository to the last committed state:
git reset --hard

will remove untracked files

git clean -d -x -f 
-d directories
-x files ignored by git
-n for dry-run
-i interactive mode
-f force
-X Remove only files ignored by Git. This may be useful to rebuild everything from scratch, but keep manually created files.

Checking used port/application in windows command

Checking which application is using a port:

• Open the command prompt
• Type netstat -aon | findstr [port_number]
• If the port is being used by any application, then that application’s detail will be shown. The last column of the list, is the PID of that application. Take a note of the PID.
• Type tasklist | findstr [PID]
• You’ll be shown the application name that is using your port number.

Checking which port is being used by a application:

Exactly the reverse of the above steps.

• Open the command prompt
• Type tasklist | findstr [application_name]
• Make note of the PID (second column) from the details shown.
• Type netstat -aon | findstr '[PID]'
• You’ll be shown the application detail and the corresponding port to which it is listening.

no message body reader has been found for class contenttype application/json, Apache CXF rest client

The error means it needs a JSONProvider or a JSON parser in its code

Option 1: Create a JSON parser

WebClient client = WebClient.create(ENDPOINT_URL);
Response r = client.accept("application/json").get();       
MappingJsonFactory factory = new MappingJsonFactory();
JsonParser parser = factory.createJsonParser((InputStream)r.getEntity());
YourModelClass model= parser.readValueAs(YourModelClass.class);

Option 2: Register a JSON provider

List<Object> providers = new ArrayList<>();
providers.add(new JacsksonJsonProvider()); // or you can register your own provider
WebClient client = WebClient.create(ENDPOINT_URL,providers);

Microservices cheat sheet at a glance

Config server:
Provides centralized, externalized, secured, easy to reach source of application configuration. Designates a centralized server to serve-up configuration information ( configuration read from source control)
Clients connect over http and retrieve their configuration settings, in addition to their own, internal sources of configuration.

    Configuration options:

• Package configuration files with application, if any changes in configuration, it requires build and restart
• Configuration file in a common file system, Unavailable in cloud
• Use environmental variable

                - Done differently on different environment
- If large hard to manage and may duplicate occurances

• Use a cloud vendor specific solutions

      - Coupling application to a specific environment, ex cloud foundry

Other challenges to consider
- Large number of microservics
- Dynamic updates
- version control

desired configuration server would be:
- platform/cloud independent solutions
- centralized
- dynamic
- controllable (scm)
- passive (self registering)

What if configuration sever is down?
      - Spring cloud config server should typically run on multiple server instances
      - Client application can control policy of how to handle missing config server
- spring.cloud.config.failFast=true, default is false
      - config server override the local config, probably there will be a policy to backup config.

Spring cloud bus:
provides simple way to notify clients to config changes

Service Discovery

• Microservices architectures result in large numbers of inter-service calls which is very challenging to configure
• How can one application easily find all of the other runtime dependencies? Manual configuration impractical

Service discovery provides a single lookup service, client register themselves, discover other registrants. Its like mainlining an automatic dynamic list.
Example: Eureka, Consul, Etcd, Zookeeper, SmartStack etc

Typically runs in a multiple server. Client sends heart beat. Multiple profiles.

Client-side load balancer:

Traditional load balancers are server-side components

• Distribute incoming traffic among serveral servers
• Software: Apache, Nginx, HA Proxy: hardware F5, NSX, BigIP

Client-side load balancer, client side load balancer select which server to call
                Based on some criteria
                part of client software
                Server can still employ its own load balancer
Why?? some may be unavailable, some slower and some further away

Spring Cloud Netflix Ribbon is a part of Netflix OSS family a client side load balancer
        - Automatically integrates with service discovery (Eureka)
        - Build in failure resiliency (Hystrix)
        - Caching / Batching
        - Multiple protocols (HTTP, TCP, UDP)
        - Spring cloud provides an easy API Wrapper for using Ribbon

Ribbon gets the server list from a static source defined in application.yml or from service discovery
How Ribbon decides

• Ping: used to test if server is up or down
• Spring cloud default - delegate to Eureka to determine if server is up or down.
• Ribbon load balancer itself: The load balancer is the actual component that routes the call to the server in the filtered list
• Several strategies available, but they usually defer to a rule component to make the actual decisions
• Spring cloud's default : ZoneAwareLoadBalancer

Feign:

• Declarative REST client from Netflix
• Allows you to write calls to REST services with no implementation code
• Alternative to RestTemplate 
• Spring cloud provides easy wrapper for using Feign

Cascading Failure:

Hystrix

• Hystrix part of netflix  oss
• Detects failure conditions and opens to disallows further calls
• Identify fallback, fallback can be chained
• Automatically closes itself after interval

Circuit Breaker: Hystrix

• Closed when operating normally
• Open when failure is detected
• Failure definition flexible , exception thrown or timeout exceeded over time period.
• Definable "Fallback" options
• Automatically re-closes itself

Api Gateway:

     - Build for specific client needs ("facade")

     - Reduces remote call

     - Routes calls to specific servers

     - Handles security / SSO

     - Handles caching

     - Protocol Translation

     - Optimizes calls / Link Expansion

     Spring cloud Zuul

           - Zuul JVM-based router and load balancer

           - Can be used for many API Gateway needs

           - Routing - send request to real server ( Reverse proxy)

     Is Zuul an API Gateway?

  - Zuul is a tool for creating an API Gateway

  - Specifically routing

              What parts are missing?

   - Caching, Protocol translation, resource expansion / Link resolution

       Spring caching abstraction:

Annotate method with @cachable

Describe cache and key

Define a cacheManager

Backed by synchronizedMaps, EHCache, Gemfire, etc.

       Some shortcoming

- Cache value might expire or might not exist after some time.

       ETags: shallow etag, intended to save bandwidth not server work, network load

Modern http-based caching better than expires

- client requests resources

- server returns resource with etag

Hash value calculated from content

- Client sends if-none-match header with etag value whenever requesting the same resource

- server calculates new hash

if it matches, return 304

if not, return 200 new content, new Etag

       

         RestTemplate does not have caching built in but HttpClient does!

         Fiegn doesn't not have caching capability, consider creating your own aspect for use with Fiegn

m2e error in MavenArchiver.getMenifest()

If you see the below error in pom.xml

Description Resource Path Location Type
org.apache.maven.archiver.MavenArchiver.getManifest(org.apache.maven.project.MavenProject, org.apache.maven.archiver.MavenArchiveConfiguration) pom.xml

you may use the below method to resolve this issue.

From Help > Install New Software..

Add the below repository and install m2e archiver

http://repo1.maven.org/maven2/.m2e/connectors/m2eclipse-mavenarchiver/0.17.2/N/LATEST/

After installing then update the maven project forcefully

Right click project -> Maven -> update project ( don't forget to check force option)

Prune remote old deleted branches

You may find you have a ton of old remote branches hanging around. Why not prune them each fetch

So follow the screen shot below. Windows -> Preferences-> Team -> Git -> Configuration

How to resolve a conflict when you try to create a pull request to review your code to master

You have two options to resolve the conflict:
1) Resolve them without a pull request.
To do this, you would checkout the master branch, and then pull in the release branch. This is effectively the solution that Bitbucket Server give you when you ask for more information on how to solve the conflict.
git checkout master
git pull origin featureBranch1
resolve merge conflicts
git push

2) Resolve them with a pull request
To do this, you would create a branch off the tip of master, pull in the release branch and create a pull request from that branch to master. This is the best option if you don't have permission to push directly to master.
git checkout master
git checkout -b resolve-conflicts-branch
git pull origin featureBranch1
resolve merge conflicts
git push -u origin resolve-conflicts-branch
create pull request

ObjectClosedException: DSRA9110E: ResultSet is closed

When you use WebSphere Application Server with an IBM DB2 database, this exception could occur:
Invalid operation: result set is closed. ERRORCODE=-4470, SQLSTATE=null
To avoid such exceptions, you must add custom properties in WebSphere Application Server at the Application Center data source level.
Procedure

• Log in to the WebSphere Application Server administration console.
• Select Resources > JDBC > Data sources > Application Center DataSource name > Custom properties and click New.
• In the Name field, enter allowNextOnExhaustedResultSet.
• In the Value field, type 1.
• Change the type to java.lang.Integer.
• Click OK.
• Click New.
• In the Name field, enter resultSetHoldability.
• In the Value field, type 1.
• Change the type to java.lang.Integer.
• Click OK and save your changes.

The possible values of resultSetHoldability are:
1 (HOLD_CURSORS_OVER_COMMIT),
2 (CLOSE_CURSORS_AT_COMMIT).

Debugging maven project, breakpoints don't work

By default, Maven runs your tests in a separate ("forked") process. You can use the maven.surefire.debug property to debug your forked tests remotely, like this:

mvn -Dmaven.surefire.debug test

For more information
http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html


In eclipse got to
Right click on the project -> Run As - > maven build ... -> build configuration

type "-Dmaven.surefire.debug test" in goal section

and add parameter forkCount = 0

Apply and hit debug